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RECOMMENDATION QUALITY AND PLACEMENT SUCCESS 


CHAPTER 1 


THE PROBLEM IN GENERAL TERMS 


O RECOMMENDATIONS mean anything? 
Do successful candidates, in general, 
have better recommendations than unsuc- 
cessful candidates? Such questions are of 
interest to all concerned with the place- 
ment process. Recommendations are 
supposed to be over-all estimates by teach- 
ers and former employers, people who 
should be in a position to know the can- 
didate’s qualifications, so there is suff- 
cient reason to presume that recom- 
mendations are important in the process 
of securing a position, and yet, actual con- 
tact with the placement process has given 
many people, candidates and employers 
alike, an uneasy feeling about the signifi- 
cance of recommendations. Some will ar- 
gue that recommendations are influential 
and some will argue that they are not, but 
very few argue with much confidence. Em- 
ployers may wonder how much weight to 
put in recommendations, but in the ac- 
tual practice of employing candidates 
they must answer the question for them- 
selves at least. And the answer is written 
in what they do. Therefore it seemed im- 
portant to take actual job-situations and 
to investigate the relation between an esti- 
mate of the quality of recommendations 
and success attained in securing teaching 
positions. This study is focused on that 
relation. 

In general, the problem demands a 
comparison of the quality of recommen- 
dations of successful candidates with that 
of unsuccessful candidates. Do successful 
candidates have better recommendations 
than unsuccessful candidates? Immediate- 
ly two questions arise. What is a success- 


ful candidate? With respect to what are 
recommendations to be classed as being of 
better quality? It is easy to see that success 
is a relative matter and means one thing 
when the position carries a salary of 
$2400 and quite another thing when the 
salary is $1600. It is known, for instance, 
that many candidates unsuccessful in se- 
curing $2400 positions have had little 
difficulty in securing $2000 positions, 
whereas other candidates successful in se- 
curing $1600 positions have had little 
fortune with $2000 positions. The concept 
success must be refined to read success 
with respect to position X. If that is true, 
obviously, the unsuccessful candidates 
compared with the successful candidate 
must be unsuccessful with respect to the 
same position. In the present study each 
comparison of successful candidates with 
unsuccessful candidates has been made 
within the context of a given position. 
There are several characteristics of 
recommendations which might have been 
investigated, but in the research reported 
here primary attention has been given to 
the quality of the recommendation, by 
which is meant the enthusiasm for the 
candidate expressed by the writer as ex- 
hibited in the favorableness of the modify- 
ing terms employed. The recommenda- 
tions were analyzed and evaluated with 
respect to this characteristic by a process 
which gave each recommendation a score. 
Then the recommendations of the suc- 
cessful candidate were compared with 
those of his unsuccessful competitors for 
the same position, to discover whether 
there was any association between recom- 
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2 WELLS HARRINGTON 


mendation quality and placement suc- 
cess. 

It was recognized that the findings 
might be influenced by extra-recommen- 
dation factors that had interfered in such 
a way as to conceal an underlying relation 
between placement success and recom- 
mendation quality. Therefore a study 
was made of each job-situation individ- 
ually, on the basis of which certain job- 
situations were selected as being reason- 
ably free from the influence of such in- 
terfering factors. For this reduced num- 
ber of selected job-situations the com- 
parison of the recommendations of suc- 
cessful candidates with those of their 
unsuccessful competitors was repeated. 

Inasmuch as mass data were employed, 


statistical techniques were required. For 
the most part analysis of variance meth- 
ods were relied upon, the use of which 
effected a pooling of the comparisons 
made within the context of the individual 
job-situations. 

The job-situations studied involved 
positions to teach the regular academic 
subjects on the secondary level. The can- 
didates were all registrants in the Office 
of Placement Service of Teachers College, 
Columbia University, who were nomi- 
nated for the vacant positions in response 
to inquiries addressed to the Office during 
the academic year 1936-37. Fifty-eight 
positions were investigated for which 183 
candidates were nominated, there being a 
total of 1035 recommendations. 





CHAPTER 2 


‘THE SELECTION OF THE JOB-SITUATIONS 


HE JOB-SITUATIONS studied in this in- 
Giger involved positions to 
teach the regular academic subjects on 
the secondary level: English, French, Ger- 
man, Italian, speech, history, social stud- 
ies, mathematics, and the sciences, or any 
combination of these subjects. Positions 
to teach in junior high schools, senior 
high schools, and junior colleges, whether 
public or private, were accepted. Posi- 
tions as critic teachers in the training 
schools of teacher training institutions 
were also included. No position was ac- 
cepted unless it was a position to teach 
at the secondary level, defined as grades 
7 through 14. 

All of the job-situations were located 
through inquiries or requests for the 
nomination of candidates which had 
been received by the Office of Placement 
Service of Teachers College and which 
had been closed during the period of a 
year, from July 1, 1936 to July 1, 1937. 
The positions connected with the in- 
quiries, for the most part, required the 
candidate to begin work either in Sep- 
tember, 1936, or in September, 1937. 
Some positions were for 1937 summer 
sessions, however, and in one or two 
cases the work started in the middle of 
the academic year. 

All of the inquiries that were received 
and were closed during the period men- 
tioned had been filed alphabetically by 
name of the community in which the 
position was located and by state. These 
inquiries were reviewed and all those 
regarding secondary school positions in 
the subjects enumerated were taken out. 
From among these were selected all of 
the job-situations in which the employer 


had either chosen a Teachers College 
candidate or had definitely indicated his 
preference among Teachers College can- 
didates. Whenever there was clear indica- 
tion of the preference of the employer 
for a given candidate from among the 
several suggested by the Office of Place- 
ment Service, that candidate was con- 
sidered successful for purposes of this 
research, regardless of whether he was 
offered the position or whether he ac- 
cepted it. 

All of the correspondence and records 
relating to the inquiry and to the regis- 
trants connected with it in any way were 
studied in order to determine just which 
ones were bona fide candidates for the 
position. On the inquiry record form 
used in the Office all possible candidates 
for nomination for a vacant position are 
listed. The form is used as a practical 
memorandum to refresh the memory of 
the executive during the process of assist- 
ing an employer. It is not a record of 
what occurred, written after the conclu- 
sion of the process. Therefore it was 
necessary to check against other records 
to be sure that any given individual had 
been thought of by the employer as a 
candidate for the position. The form 
itself provided the first check because 
the date of dispatch of credentials to the 
employer was usually entered. Candi- 
dates are requested to return a memoran- 
dum to the Office indicating whether 
they have or have not applied for the 
position. That provided a second check; 
but not every candidate is careful to re- 
turn the memorandum indicating his de- 
cision, and some employers ask the Office 
not to have the candidates send letters 





1 
nee 


bE: 


ate: 
a eee 


Fea IR eee reg 


fee 





Seige tee X 
3 Pv 








4 WELLS HARRINGTON 


of application in which case there is no 
memorandum to be returned. In the in- 
dividual candidate’s file, however, there 
is a record of employers to whom his 
credentials have been sent together with 
the dates of their dispatch. For a third 
check this record was examined for every 
candidate and the date the credentials 
were sent out was checked with the date 
the inquiry was received and the date 
the position was filled. Reading the cor- 
respondence provided a fourth check in- 
asmuch as the names of the candidates 
who had applied and were being con- 
sidered were usually mentioned, From 
four sources, then—the inquiry record 
form, the notice from the candidate indi- 
cating whether he had applied for the 
position, the record of credentials sent 
to employers, and the correspondence— 
a check was obtained on whether the 
candidate had applied for the position 
and whether his credentials had been 
sent to the employer. Unless there was 
positive evidence that the candidate had 
applied and that his credentials had been 
sent to the employer, the candidate was 
eliminated from consideration in regard 
to that job-situation. 

In some cases a candidate who had ap- 
plied for the position and whose creden- 
tials had been sent lost interest when the 
salary was learned. Such candidates were 
eliminated. Two candidates were elimi- 
nated from one job-situation, in spite of 
the fact that one of them was successful, 
because there was no evidence that their 
credentials had been evaluated by the 
employer. Both candidates had been 
interviewed by the employer, but there 
was no record of their credentials having 
been sent out. 

Inasmuch as all of the research was 
done after the positions were filled, the 
sets of credentials studied sometimes 
contained recommendations written after 


the time at which the candidate’s creden- 
tials were reviewed by the employer. 
Therefore the date of every recommenda- 
tion of each candidate was checked with 
the dates connected with the job-situa- 
tion. Most of the recommendations were 
found to have been written before the 
date of the inquiry. If a recommendation 
had been written after that date, its date 
was checked with the date on which the 
credentials had been sent to the em- 
ployer. If a recommendation had been 
written after that date, the correspond- 
ence was re-read to see if there was men- 
tion of the forwarding of additional rec- 
ommendations—a frequent occurrence. 
Comparison of the date of the recom- 
mendation with the date the position 
was filled provided a negative check. 

Finally the credentials of every can- 
didate as well as the records pertaining to 
every job-situation were checked to be 
sure that all of the information needed 
for the research was complete. A total of 
96 inquiries were found to have records 
warranting further investigation of the 
job-situations. 

These 96 inquiries represent a selec- 
tion from all of the inquiries, of the type 
studied, received by Teachers College 
throughout the years. Membership in 
that sample is determined on the basis 
of the year in which the inquiry was re- 
ceived, so it is not a random sample. Yet, 
probably, few would maintain that it is 
a biased sample. The job-situations rep- 
resented by the inquiries received by 
Teachers College requesting the nomina- 
tion of candidates qualified to teach cer- 
tain secondary school subjects probably 
do not vary much from year to year. 
There is no reason to suppose that the 
year 1936-37 was atypical in the calibre 
of the candidates available, in the char- 
acter of the recommendations written 
for those candidates, in the nature of the 
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positions to be filled, or in the reactions 
of the employers toward recommenda- 
tions. Nevertheless, the dependability 
with which conclusions based upon a 
study of 1936-37 job-situations can be 
applied to the job-situations of other 
years is a matter of judgment. The au- 
thor is willing to leave the judgment 
to the reader, though it seems to him that 
conclusions for 1936-37 would be ac- 
ceptable for a period of years before and 
after that time. 

For similar reasons the 96 job-situa- 
tions cannot be considered a random 
sample of the job-situations referred to 
the placement offices of teacher train- 
ing institutions generally. To the extent 
that Teachers College is a unique insti- 
tution, it might even be considered a 
selected sample. However, inasmuch as 
it is unlikely that the qualitative aspect 
of the relation between recommendation 
quality and placement success varies 
greatly from one type of job-situation to 
another, findings concerning that rela- 
tion may be considered important. Again 
the final judgment is left to the reader 
as to the extent to which conclusions 
based upon job-situations referred to the 
Teachers College placement office should 
be applied to job-situations generally. 

Using cards to facilitate the process, 
the job-situations, and thereby the can- 
didates, were selected in the following 
manner. From among the 96 job-situa- 
tions one was selected by chance. Then 
a second job-situation was selected by 
chance. If there was no duplication in 
the names of the candidates listed for the 
two positions, the second job-situation 
was retained. If one or more candidates 
appeared as nominees for both positions, 
the second job-situation was rejected. 
Then a third job-situation was selected 
by chance. The list of nominees for this 
position was compared with the lists 


for both previous positions, and so on. 
Finally 58 of the 96 job-situations had 
been retained and 38 had been rejected. 
For these positions there were altogether 
183 nominees whose candidacies could be 
used for the purposes of this investiga- 
tion. Every one of these candidates ap- 
pears in this study in connection with one 
job-situation only. No candidate can be 
found as a nominee for more than one 
position and likewise no candidate can 
be found in both the successful and un- 
successful groups. The statistical methods 
used for the comparison of the recom- 
mendations of successful candidates with 
those of their unsuccessful competitors 
rest upon the assumption that the data 
in the two groups are independent 
whereas the presence of members com- 
mon to both groups would have intro- 
duced a degree of correlation. 

It is important to note that all of the 
58 job-situations were selected by a 
chance process and that none of the re- 
jected job-situations was discarded for 
any reason other than that it had not 
been previously selected by chance. 
There was no special basis for selecting 
either the retained or rejected job- 
situations, and in neither operation was 
bias involved. All of the job-situations 
available under the requirements of the 
statistical methods employed were re- 
tained for use in the comparisons to be 
made. 

Research of this type is usually carried 
on in order to discover whether a certain 
relationship is true in general. Whether 
the relationship is true in particular is 
of interest because it throws light on the 
larger question of whether it is true in 
general. Where mass data are involved, 
making the connection between the par- 
ticular case and the general situation 
is a matter of statistical inference. A defi- 
nite hypothesis about the population or 
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6 WELLS HARRINGTON 


the condition in general is evaluated, 
i.e., accepted or rejected, in the light of 
observed findings for a sample drawn 
from the population. To make this evalu- 
ation the sample must be secured by an 
established procedure, the usual proce- 
dure being designed to secure a random 
sample. When, however, the investigator 
does not have a random sample but in- 
stead has all of the cases available for 
study of the particular question at hand, 
as is the case in this research, it is neces- 
sary to assume a hypothetical population 
of which the ¢ases available for study may 
be considered a random sample. ‘Then 
generalization is made from the groups 
of cases studied to the hypothetical popu- 
lation. The usefulness of the conclusion 
regarding the hypothesis depends upon 
the character of the hypothetical popu- 
lation, especially upon the degree of its 
similarity to existing populations. 

Such a hypothetical population for this 
study could be thought of as being com- 
posed of a very large number of candi- 
dates similar to the candidates of the 
group studied, for whom recommenda- 
tions were written by persons writing 
similarly to those who wrote the recom- 





mendations studied, who were nomi- 
nated for positions similar to those of the 
58 job-situations studied, and who were 
divided into the categories successful and 
unsuccessful by’ employers activated by 
motives similar to the motives of the 
employers of the job-situations studied. 
Hypotheses about this defined popula- 
tion can be tested against the findings 
for the candidates of the 58 job-situa- 
tions. For example, the hypothesis that 
there is, in general, no difference between 
the recommendations of successful candi- 
dates and those of their unsuccessful com- 
petitors with respect to recommendation 
quality can be tested and accepted or 
rejected. The implications for other pop- 
ulations of candidates for teaching posi- 
tions depends upon the similarity of 
those populations to the defined hypo- 
thetical population. This again is a 
matter of judgment and is for the reader 
to determine. It is the author’s opinion 
that conclusions made with reference to 
the hypothetical population here de- 
scribed have a substantial degt@e of valid- 
ity for candidates for teaching positions 
in general. 





CHAPTER 3 


THE PROCEDURE FOR SECURING THE DATA FROM THE RECOMMENDATIONS 


ost of the recommendations studied - 


were written in response to the 
following instructions taken from the 
form used for requesting recommenda- 
tions provided by the Office of Placement 
Service: 


“Please write on this form your estimate 
of the registrant whose name appears above 
and return the form directly to the Office 
of Placement Service. What you write may 
be sent by the Office to prospective em- 
ployers. If you wish to state anything for 
the guidance of the Office alone, please put 
it on the back of this form. 

“The Office desires to have as registrants 
not only present graduate students of 
Teachers College but men and women who 
have successfully completed graduate work 
at Teachers College and who by training 
and experience are qualified for positions 
of responsibility.* 

“How do you rate this registrant's intelli- 
gence and personality? Please state what you 
know of the registrant’s accomplishment as 
a student; as a worker in a specific position; 
and his probable future success. 

“* Only the well qualified should be 
recommended for positions. A statement of 
the Office’s policy and practice will be sent 
upon request.” 


A written recommendation is a more 
or less extended declaration by the 
writer in which he relates facts as he 
knows them and expresses his judgment 
concerning the candidate and his quali- 
fications for a specified kind of teaching 
position. It appears as a series of sen- 
tences, but for purposes of this study, it 
might better be thought of as a series of 
statements about the candidate’s quali- 
fications, Clearly one sentence may em- 
body many such statements. For example, 
“Miss —— has a very pleasing person- 
ality, is open, frank, and honest.” This 


sentence represents four statements of 
evaluation of the candidate. On the 
other hand, one statement may be spread 
over many sentences. For example, 
“Among the many instances of his suc- 
cess here I might mention his taking 
complete charge of a large intermediate 
algebra class of 39 pupils in the second 
semester. He had assisted with this class 
the first semester. The work during the 
second semester was entirely under his 
supervision.” ‘This is essentially one state- 
ment about the general character of the 
candidate’s classroom teaching spread 
over three sentences. There are many as- 
pects of the candidate’s qualifications, 
personal and otherwise, that may be 
commented upon. Later in this chapter 
a detailed list of the possibilities will be 
presented in the form of an Inventory. 

Recommendation quality is the char- 
acteristic studied in this research. The 
term is used to refer to an estimate of the 
enthusiasm with which the candidate is 
recommended made upon the basis of 
the favorableness of the modifying terms 
employed by the writer to describe the 
candidate’s qualifications. The author is 
thoroughly aware of the fact that favor- 
ableness of modifying terms employed is 
not the only basis for estimating enthu- 
siasm of recommendation, nor is he in- 
clined to argue that it is the best basis. 
It is an acceptable basis, however, and, 
as will be shown later, an adequate basis 
for the statistical treatment of the data. 
Undoubtedly individual writers of rec- 
ommendations vary in their tendency to 
use modifiers and also in the liberality 
with which they use superlatives, but 
the individual departures from the usual 
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in the pooling process of the statistical 
method tend to cancel each other so that 
the effect on the end result is more or 
less random and probably slight. ‘There 
is no reason to believe that the atypical 
writer has more often written for either 
success group. It is reasonable to sup- 
pose that a given professor might be lav- 
ish or sparing with superlatives. Although 
the estimates of the candidates of the pro- 
fessor’s department might thus be af- 
fected, the effect would not be partial 
to either successful or unsuccessful can- 
didates, and, therefore, the comparison 
of the two groups would not be affected. 

The author is thoroughly aware of the 
fact that recommendation quality is not 
the only constituent of recommendation 
excellence that bears investigation. Other 
characteristics of recommendations seem 
to offer promising possibilities. It is very 
likely that many writers place their pri- 
mary discrimination among those for 
whom their recommendations are writ- 
ten upon other bases than the use of 
modifying terms. In all likelihood the 
selection of qualifications commented 
upon is equally important. The author 
thinks of recommendation quality as 
defined as only one of many acceptable 
constituents of recommendation excel- 
lence worth studying. 

The process of analyzing recommenda- 
tions and securing an estimate of recom- 
mendation quality as a score, in general, 
consisted of three parts. First, the recom- 
mendation was divided into a series of 
statements about separate subjects. The 
statements were classified by referring 
them to an Inventory (see pages 14 ff.). 
Second, each statement was rated for 
quality on the basis of the favorablenéss 
of the modifying terms found in the 
statement. The rating of modifying terms 
was accomplished by referring them to a 
Quality Rating Scale (see pages 27 ff.). 


Third, the ratings for individual state- 
ments were averaged to obtain a score 
for the recommendation. 

Two types of material are included by 
writers in recommendations: (1) opinions 
and facts which can be rated on the basis 
indicated for all types of positions and 
employers, which will be called evalu- 
ations, and (2) facts and opinions which 
cannot be rated uniformly but must be 
rated in relation to a given position and 
a given employer, which will be called 
non-ratable information. Opinions pre- 
dominate in the first type of material, 
factual statements in the second. Evalu- 
ations are illustrated by the comment, 
“He is very cooperative.” This statement 
evaluates the candidate by attributing to 
him a certain personality trait, cooper- 
ativeness, and a certain degree of that 
trait, the degree expressed by the word 
very. Irrespective of the type of school, 
the subjects to be taught, the kind of - 
community, and the predilections of the 
employer, such a statement can be rated 
as being in the candidate’s favor to the 
degree represented by the modifier very. 
The rating does not depend upon the 
position for which the individual is a 
candidate. 

Non-ratable material, on the other 
hand, cannot be rated except in relation 
to a specific position. Consider a state- 
ment that the candidate is a major in 
mathematics. Whether the statement is in 
the candidate’s favor depends upon the 
subject to be taught. Or take an opinion 
that the candidate will do his btst work 
in an urban community, Again the rat- 
ing depends upon the type of community 
in which the position is located. No at- 
tempt was made to rate statements of the 
second type for several reasons. Factual 
statements, which form the bulk of the 
material, usually repeat information 
which is more complete and dependable 
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in the biographical section of the cre- 
dentials. It is believed that most em- 
ployers refer to the biographical section 
for such information. Approximate 
equalization of the effects of the more im- 
portant of these factors is provided for 
in the treatment of extra-recommenda- 
tion factors described in Chapter 5. In 
addition factual statements of the kind 
referred to here are not expressed in 
degree and do not contain modifying 
terms. But the principal argument against 
including the second type of material in 
the estimate of recommendation quality 
was that the rating would have had to be 
in relation to the position in question, 
whereas it was absolutely necessary, while 
the recommendations were being rated, to 
keep study of the recommendations and 
study of the job-situations separate. If the 
position and the candidates for it had 
been studied at the same time, it would 
have been impossible for the author, who 
did the rating, to have avoided knowing 
who was the successful candidate and 
thereby becoming prejudiced in the rat- 
ing. 

In terms of procedure the principal re- 
quirements of the analysis of the recom- 
mendations called for: 


1. Identification and recording of the job- 
situation, the candidate, and the recom- 
mendation in a manner that would 
keep the identity of the successful 
candidate unknown to the analyzer. 

2. Recording of the date the recommenda- 
tion was written. 

3. Classification of each individual state- 
ment by the writer of the recommenda- 
tion as to the subject of the comment. 

4. Rating of each evaluation for quality 
of comment. 

5. Recording of the classification of each 
statement with the item number from 
the Inventory. 

6. Recording of the estimate of quality or 
quality rating for each evaluation. 


In order to make the procedure con- 


sistent from one recommendation to an- 
other, an Inventory of possible subjects 
of comment by writers of recommenda- 
tions, a List of Equivalent Trait-Names 
and Phrases, and a Quality Rating Scale 
were constructed to be used as guides 
by the author. To fulfill its function the 
Inventory needed to include classifica- 
tions describing all of the personal char- 
acteristics or traits of a candidate in addi- 
tion to classifications for the variety of 
qualifications peculiar to the vocation of 
teaching as well as classifications for non- 
ratable information. The Inventory 
needed to meet these four criteria: 

1. The list of items should be complete 
so that there would be a classification 
for every recommendation statement 
found. 

2. Overlapping among items should be 
reduced to a minimum so that a 
recommendation statement would 
clearly belong in one classification and 
clearly not belong in any other. 

3. Appropriate headings should be chosen 
and each item properly located under 
the headings (so that, if undertaken, 
study of the relation between scores by 
headings and placement success might 
be meaningful). 

4. The items should be defined in terms 
commonly found in recommendations 
in order to retain as much as possible 
of tHe original character of the data. 

In the construction of the section on 

Personal Characteristics it was recognized 
immediately that in any absolute sense 
it is impossible to avoid overlapping of 
meaning. To most students of human 
personality, each of the 17,953 trait- 
names listed by Allport and Odbert 
(1, p. vi)! represents a unique, distinct, 
and definite facet of human character not 
wholly like any other one of the entire 
list. Even this list is replete with over- 
lapping of meaning although there is no 


1 The first number indicates the reference as 
listed in the Bibliography; the second number 
indicates the page cited. 
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true duplication of meaning. If then a 
list of 63 trait-names and phrases de- 
scriptive of personal characteristics is 
presented, necessarily there is appreciable 
overlapping. The practical research prob- 
lem at this point was to construct a list 
which was short enough to be useful, and 
which, given the number of traits in- 
cluded, involved the least amount of 
overlapping of meaning in the items. A 
minimum requirement was the elimina- 
tion of overlapping which went so far as 
to become duplication of meaning. 

In this connection it should be clearly 
appreciated that the definition used in 
the Inventory to describe an aspect of a 
candidate’s qualifications is only a par- 
tial definition of the thing described. 
I'he only complete definition would con- 
sist of the Inventory item plus the inter- 
pretations that have actually been used 
by the author in analyzing the recom- 
mendations. Indeed, it is even possible 
that an item which has been poorly de- 
fined in the Inventory may yet be highly 
effective in the way it has operated as a 
euide in the analysis of the recommenda- 
tions. 

It was essential that the Inventory 
items be defined in terms commonly em- 
ployed by writers of recommendations. 
Thus, ascendance and extroversion might 
very well have been used as classifications 
under Personal Characteristics except 
that candidates are almost never de- 
scribed in such terms in recommenda- 
tions. Among 1035 recommendations the 
author cannot remember a single use of 
ascendance, extroversion, or similar psy- 
chological terms. Translating a recom- 
mendation statement into such terms 
would interpose an additional step be- 
tween the data in the form in which ‘it 
was found and the data as recorded. It 
seemed advisable to avoid that additional 
separation. 


Because all aspects of personality have 
important bearing on the effectiveness 
of the teacher, the section on Personal 
Characteristics attempts to present, with- 
in the limits imposed by the employment 
of commonly used trait-names, an outline 
for a complete description of an indi- 
vidual. It is hoped that the trait-names 
selected represent definable and deline- 
able traits or personal characteristics 
which are relatively free from overlap- 
ping of meaning. 

The construction of the Inventory in- 
volved five inter-related operations that 
were interwoven in the actual process. 
First, an exhaustive list of trait-names, 
phrases, etc., representing traits and 
qualifications had to be prepared. Sec- 
ond, it was necessary to reduce this list 
of trait-names and phrases to a list of 
traits and qualifications, i.e., to decide 
which trait-names, phrases, etc. were 
variant designations for the same trait 
or qualification and which groups of 
designations represented different traits 
or qualifications. This involved the diffi- 
cult task of deciding whether a described 
quality was to be considered a trait or 
qualification, or whether it was merely 
to be considered an aspect of another 
quality which was the trait or qualifica- 
tion. Incidental to the second operation 
was the third, the selection of a desig- 
nation to represent the trait or qualifica- 
tion. The fourth operation was the selec- 
tion of headings, and the fifth, the loca- 
tion of items under the proper headings. 

In the beginning, scores of recom- 
mendations were read to discover, in gen- 
eral, what traits and qualifications were 
commented upon by writers of recom- 
mendations. Then each recommendation 
was divided into statements about single 
items of the candidate’s qualifications, 
and the essence of the statement recorded 
on a separate card. At first no attempt 
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was made to group together words or 
phrases referring to the same item, but 
a new word or phrase was assumed for 
the time being to represent an additional 
item. The dictionary and Roget’s The- 
saurus were consulted to provide other 
words and phrases, frequently synonyms 
‘and antonyms, for the same item. 

Then the literature on the subject of 
traits related to teaching success was 
reviewed. The different types of studies 
may be classified on a variety of bases, 
each basis being with respect to a differ- 
ent aspect of the study. Some investigators 
studied traits related to teaching suc- 
cess; others, traits related to failure. 
Some attempted to predict success or 
failure; others analyzed status already 
achieved. Some studies were confined to 
personality traits; others dealt with all 
kinds of qualifications, personal and 
otherwise. In-some studies the investiga- 
tion of traits was only part of a broader 
inquiry. Some investigators used a check- 
list which suggested the trait-names to be 
appraised; others allowed a free response 
to a general question. In some studies the 
respondents were instructed to rank the 
traits in order of importance; in others 
they were asked merely to indicate which 
traits were considered important. The 
purpose of most studies was to establish 
the relative importance of traits, which 
could be accomplished by ranking traits 
either according to the percentage of re- 
sponses in which the trait was considered 
important, or by averaging the ranks 
given to the trait. The purpose of other 
studies was merely to prepare a list of 
traits considered important by a given 
body of opinion. Obviously, the purpose 
and the method of collection of data de- 
termined the type of statistical procedure 
used. Administrators, supervisors, princi- 
pals, and college teachers of education 
were the people usually polled, although 


there are several studies of student opin- 
ion. Investigations of the traits listed in 
letters of reference forms used by place- 
ment offices and school systems were also 
reviewed. 

Individual trait-names from these lists 
were found to duplicate trait-names al- 
ready secured from the perusal of recom- 
mendations. None of the lists as a whole, 
however, could be used because each had 
one shortcoming or another with respect 
to use in the investigation reported here. 
The classifications of many of the lists 
show considerable amounts of overlap- 
ping among the items. Some of the lists 
include large headings only. Personality, 
for instance, was listed as one item. In 
dealing with personal characteristics, ap- 
parently none of these authors had con- 
ceived of the more significant problem as 
the collection of facts about traits rather 
than trait-names, 

Several scales for the measurement of 
teaching efficiency have been constructed. 
These were reviewed in order to check 
the completeness of the list of trait-names 
secured from the perusal of the recom- 
mendations. A teacher rating scale may 
be defined as a list of traits which when 
rated yields a score which is related to 
teaching success. This list of traits is not 
necessarily an inventory of all aspects of 
an individual related to teaching success 
nor of all aspects of a candidate likely 
to be commented upon in recommenda- 
tions. Of recent scales, the Torgerson 
Diagnostic Teacher Rating Scale of In- 
structional Activities (18), for instance, 
is confined to classroom teaching, which 
is but one aspect of the many commented 
upon in recommendations. The Almy- 
Sorenson Rating Scale for Teachers (3) 
on the other hand is confined to per- 
sonality traits. An impression gained 
from most scales is that the compromise 
with the practical necessity of a short 
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list of traits is considerable. As a result 
many scales include trait-names that are 
not sub-divided enough to reach the 
point where traits, as opposed to families 
of traits, are represented. A. S. Barr, re- 
‘erring tq the measurement of teaching 
ability, aptly summarized the situation 
when he said, “If we employ the quali- 
‘ies approach, there is little agreement 
among workers relative to the qualities 
essential to success in teaching, the 
amounts of, these qualities necessary for 
success, the relative importance of the 
several qualities considered essential to 
success, and how best to define these 
qualities” (5, p. 182). The word qualities 
is used by Barr appears to have the same 
meaning as traits in the present study. 

Studies have also been made of the con- 
tent of recommendations. At the Univer- 
sity of Oregon 308 recommendations 
vritten for 56 students “were carefully 
inalyzed, and a check kept on the quali- 
ties attributed to the applicant, and of 
the degree to which the candidate was 
assumed to possess these qualities” by 
\lmack (2, p. 33). At least 57 different 
qualities were found but the complete 
list is not reported. Evidently there was 
no attempt to deal with traits as distinct 
from trait-names. 

Jones (12) studied the traits found in 
,00 recommendations written for seniors 
1t Oberlin College by members of the 
faculty, and secured a list of 80 trait- 
names which were found 5 or more times 
in the recommendations. Although this 
list was secured empirically from a study 
of recommendations, it is really a list of 
trait-names and not a list of traits, Dili- 
zent and hard-working, for example, are 
eparate items, 

Dozier (10) analyzed 1132 recommen- 
lations and found 805 trait-names. She 
lid not attempt to construct an inven- 
tory by any process of reducing over- 


lapping among the 805, trait-names, but 
instead drew up an outline (or inven- 
tory) which she believed would include 
all of the qualities found. To quote, “It 
was not thought ‘desirable in this study 
to condense the number of qualities by 
combining and construing many of them 
as synonyms. The following outline pre- 
sents a Classification of all qualities listed 
in the recommendations. . . .” (10, pp. 
326-327). In the opinion of the author 
the outline is much too sketchy for use 
in this study. 

A very interesting list of vivid adjec- 
tives descriptive of personality was com- 
piled by Sheppard (16, p. 446) as a result 
of several years of placement experience, 
but the list was not intended to be a 
complete inventory. 

Vick (1g) studied the traits considered 
to be the most influential in recommen- 
dations by polling the opinion of 252 
school administrators in Illinois. from 
whom he received 160 replies giving their 
reaction to the relative importance of the 
45 traits of a scale prepared by Boyce 
(6, p. 44). 

Weidemann studied 1250 recommenda- 
tions in order to answer three questions, 
among them being this, “What is a gen- 
eral plan of classification of the content 
of the letters of recommendation?” (20, 
p. 69). From the brief description of the 
method of analyzing the recommenda- 
tions and of constructing the general 
plan of classification given in the article 
it appears that the procedure used by the 
author must have been similar to that of 
Weidemann. With his investigation as a 
background, Weidemann constructed a 
New Type Letter of Recommendation, 
essentially a rating device or score card, 
which contains an inventory which has 
much to recommend it (20, pp. 88-92). 

As an inventory of personal traits the 
Master List of Teachers’ Traits produced 





RECOMMENDATION QUALITY AND PLACEMENT SUCCESS 13 


by the Commonwealth Teacher- Training 
Study deserves special mention because 
of the thorough and painstaking work 
that went into its preparation (8, p. 18). 
A long list of trait-names defined by de- 
scriptive actions was secured by the in- 
vestigators through personal interviews 
with g7 members of the profession. 
Twenty-one judges then classified the de- 
scriptive actions under 83 traits (or trait- 
names). This list was arbitrarily reduced 
in number by combining similar trait- 
names until 25 traits remained (8, pp. 
14-19 & 51-76). ““The list of 83 traits was 
accordingly telescoped by grouping the 
items in families. Because it was known 
that 25 traits could be used to advantage 
in teacher-training institutions, the num- 
ber of families was arbitrarily fixed at 25,” 
(8, p. 63). This process of reducing the 
list of trait-names by combining those 
with similar meaning is the one used by 
the author but in the Commonwealth 
Study the process was continued until the 
arbitrary number of 25 traits was reached, 
whereas, the author had no arbitrary 
number of final traits fixed in advance. 
Ordinarily the number of traits which 
finally survive such a process depends 
upon the amount of similarity demanded 
of the trait-names to be combined. That 
being the case, selection of an arbitrary 
number of final traits is one way of de- 
termining the amount of similarity in 
trait-names to be used as a criterion; and 
if the criterion is kept uniform through- 
out the process, as seems to have been 
the case in the Commonwealth Study, 
it is a safe as well as legitimate method. 
But it is to be doubted if 25 approxi- 
mates the best number to serve in secur- 
ing a list of traits for analyzing recom- 
mendations. Neither the list of 25, traits 
nor the list of 83 traits seemed com- 
pletely satisfactory for the investigation 
contemplated in this research. The In- 


ventory constructed for this study con- 
tains traits not to be found in either list 
and divides into separate traits certain 
of the traits of the Commonwealth Study 
which in the opinion of the author con- 
tain too much overlapping. Experimental 
attitude, for example, is a trait not to 
be found in the Commonwealth Study 
list. The latter classifies interest in the 
community, interest in the profession, 
and interest in pupils all under one head- 
ing, breadth of interest. In the Inventory 
a distinction is made between tact and 
sympathetic nature, two traits put under 
the same heading in the Commonwealth 
Study list. In the latter intelligence and 
originality are under separate classifica- 
tions while in the Inventory both are 
classified under intellectual ability, Other 
examples might be cited. Since it was 
constructed for a different purpose, 
namely, to provide direction for empha- 
sis in teacher training, it is not surpris- 
ing that the Commonwealth Study list 
was not found exactly suited to the needs 
of this study. 

The Commission on Educational Lead- 
ership (14, pp. 290-291) studied recom- 
mendations written for 105 superintend- 
ents considered successful by 264 school 
board members who wrote the recom- 
mendations. It was found possible to 
tabulate the characteristics mentioned 8 
or more times under go headings. As the 
result of an empirical study, the fre- 
quency list of characteristics reported is 
valuable, but the list does not present a 
satisfactory inventory. Good disciplin- 
arian and Firm when necessary, for ex- 
ample, are separate items. 

Morrisett (13) studied 1851 letters of 
recommendation written for candidates 
for high school teaching positions by 
members of the profession and laymen 
from various sections of the country in 
order to discover what information was 
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contained in the recommendations writ- 
ten by four classes of writers: teachers of 
education, college subject matter teach- 
ers, principals and superintendents, and 
laymen. Each item was classified accord- 
ing to the Master List of Teachers’ Traits 
produced by the Commonwealth Study 
(8, p. 18) and a frequency count reported 
for the letters of the four classes of writ- 
ers. 

An excellent inventory is contained in 
an outline of a proposed brochure on 
the selection of teachers by Jarvie and 
Prescott of the Commission on Teacher 
Education (4). The authors must have 
had in mind an inventory in the sense 
that the word is used in this study and 
must have been cognizant of the neces- 
sity of making it complete. The items are 
not phrased in the terms employed by 
writers of recommendations, however, 
and items of that type are not suited to 
this study as it was conceived. One of 
the subsections (4, p. 2) is given here as 
an example: 


I. Personal and Social Characteristics. 
B. Organization of the personality. 
1. Basic values the individual is 
seeking in life. 
2. The consistency of these values 
with each other. 
3. The consistency of these values 
with cultural reality. 


Writers of recommendations do not often 
describe individuals the way psycholo- 
ists or philosophers do. 

Another excellent inventory is by Cady 
and Katsuranis (7), who elaborated a 
“working theory of personality” and pre- 
pared a list of traits for the evaluation 
of the growth of students of New College 
of Teachers College. It is evident that 
the authors had in mind an inventory 
in the sense that the word is used in this 
study and the necessity of making the 
list of traits complete. Cady and Katsu- 


ranis seemed to be particularly cognizant 
of the importance of having personality 
items represent traits as they might be 
defined psychologically, Inasmuch as it 
was designed for a different purpose, 
some of the items are not pertinent for 
the evaluation of recommendations. 

Throughout this process additional 
recommendations were constantly being 
studied. From these five sources—recom- 
mendations, the dictionary and thesau- 
rus, studies of traits related to teaching 
success, scales for the measurement of 
teaching efficiency, and other inventories 
such as that of Cady and Katsuranis— 
a considerable list of words and phrases 
describing items of a candidate’s qualifi- 
cations was built up. Then these words 
and phrases, each on an individual card, 
were grouped so that all words and 
phrases which appeared to identify the 
same item were placed together. Then 
the item was named, a definition of the 
item was constructed if that seemed neces- 
sary, headings were decided upon, and 
the location of the item under a heading 
was determined. The principal strength 
of the author’s Inventory is that it is so 
integrally related to recommendations. 
It was designed for the specific purpose 
of aiding in the analysis of recommenda- 
tions and its language is consonant with 
that purpose. The four criteria outlined 
on page 9 were constantly kept in mind 
during its construction. The Inventory 
is divided into two parts to correspond 
to the two types of comment to be found 
in recommendations. 


THE INVENTORY? 
Part One—Non-Ratable Information 


1. Type of training program undergone. 
2. Length of training program undergone. 


*In the original planning of the study the 
possibility of investigating the relation between 
several characteristics of recommendations and 
placement success was contemplated, Conse- 
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. Teaching field in which prepared and/ 


or preferred to work. 

Degree or certification status. 

Age or grade group prepared to teach 
and/or preferred to teach. 

Type of community in which the candi- 
date should work, in the opinion of the 
writer. 

Type of school in which the candidate 
should work, in the opinion of the 
writer. 

Favorable viewpoint held with respect 
to “progressive education.” 

Extreme viewpoint held with respect to 
political and social problems. 

Middle viewpoint held with respect to 
political and social problems. 
Teaching experience gained in a public 
secondary school or a training second- 
ary school of a teacher training insti- 
tution. 

Teaching experience gained in a pri- 
vate secondary school of the academy 
type. 

Teaching experience gained in a lib- 
eral arts college, teachers college, or 
university. 

Teaching experience gained in a 
special school, such as a trade school or 
business college. 

Length of teaching experience. 

Place of teaching experience. 

Subjects taught. 

Age or grade group experienced in 
teaching. 

Experience in the use of “progressive 
methods” or in a “progressive school.” 


Part Two—Evaluations 


20. 


21. 


GENERAL 


General estimate of the candidate in- 
cluding reference to both general back- 
ground and personality characteristics. 
General background. A reference to 
everything except personality character- 
istics joined together in a general state- 
ment. 





quently the analysis of the recommendations 
obtained other information than it is necessary 
to report here, and the Inventory contained 
additional items under the following headings: 
1. Relation of the Writer to the Candidate. 
2. Degree of Acquaintance of the Writer with 
the Candidate. 
§. Degree of Acquaintance of the Writer with 
the Candidate’s Teaching. 


23. 


24. 


25. 


26. 


27 


28. 


29. 


go. 


gi. 


32. 


33- 
34. 


Variety of background. Not confined to 
professional experience. 

Growth in training, experience, and 
general background. 


PERSONAL CHARACTERISTICS 


A. General 


General Estimate—General estimate of 
the candidate’s personality; general 
Statement about the candidate as a 
person or about his personality. 
Growth in Personality—Growth within 
the individual of personality traits of 
an implied favorable character. 


B. Abilities 


General Ability—Inclusive characteriza- 
tion, ability as a person. Does not refer 
exclusively to intellectual ability. Ver- 
satility. 

Special Skill—Result of innate capacity 
properly trained, such as instrumental 
skill in music. 


C. Intellectual Characteristics 


Intellectual Ability—Innate power to 
deal intelligently with abstractions, to 
reason, to think, to solve problems, to 
comprehend, to understand, to meet 
novel situations. 

Breadth of Intellectual Interests— 
Usually evidenced by possession of a 
wide variety of information such as 
comes from extensive reading to satisfy 
active intellectual curiosity. 

Critical Habits of Thought—Quality of 
mind, thought pattern. Freedom from 
prejudices, emancipation from authori- 
tarian thinking, disposition to with- 
hold judgment or conclusiogs until new 
or different evidence or reasons are 
reviewed, inclination to accept and 
hold views critically. 

Perspective in Thinking—Habit with 
regard to what is attended to. Practice 
of giving attention to both the broad 
outlines and the details in proportions 
appropriate to the situation. 
Expression of Thought—In general, no 
differentiation being made _ between 
speaking and writing. 

Oral Expression of Thought—Both in 
conversation and in public speaking. 
Written Expression of ‘Thought. 
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D. Physical Characteristics 


Physical Health. 

Physique—Body build permitting hard 
and continuous work at the occupation 
of teaching. Absence of defects. Not 
type of frame. 


E. Basic Emotional Characteristics 


37. Maturity—General impression created 


by the behavior. Not age as such. 


;8. Emotional Balance—Equability of be- 


havior, even in times of stress, based 
upon innate stability, freedom from 
unfortunate experience, and good 
mental hygiene. 


»9. Enthusiasm—Individual attitude toward 


life. High morale. Interest in life. 

Self-Confidence—Habit of depending 
upon oneself, especially in times of 
difficulty; based upon the estimate an 
individual habitually makes of himself, 
of his ability to meet problems and 
situations, and even of his capacity to 
withstand failure and its consequences. 


. Objective Intelligence—Ability to react 


intelligently, keeping personal emo- 
tional considerations from protruding 
into and influencing thinking. These 
considerations have the effect of re- 
ducing the intellectual ability of some 
individuals in certain situations. 

Ability to Make Decisions—Habit of 
making decisions without an undue 
amount of suspended judgment. 


3. Progressiveness—Attitude on the part of 


an individual towards himself which is 
conducive to growth. Includes the use 
in constructive ways of criticism, both 
positive and negative, and the desire 
for growth. 


F. Interests and Accomplishments 


‘4. Breadth of Interests—All kinds of inter- 


ests not covered in other items. 


5. Interest in Sports—Interest in athletic 


sports and games. 


). Esthetic Appreciation—Genuine _inter- 


est in what the individual conceives to 
be beautiful. The latter need not be 
generally so accepted. Refers specifically 
to color, form, shape, design, and 
texture wherever found. 

Appreciation of Music—Genuine de! 
light in listening to music. Not meant 


! 


48. 


49- 


50. 


51. 


52. 


53- 


to specify any particular kind of music. 
Travel—Quality and extent. 


G. Appearance 


General Appearance—Total effect of 
the person’s physique, posture, bearing, 
dress, manner, and physical appearance. 
Minor Aspect of Appearance—Refers to 
comment upon any one of the single 
features listed under General Appear- 
ance. os 

Statement that the candidate is a 
young man or young woman, regardless 
of the actual age. 
Manliness—Appearance and bearing of 
the type associated with the more tradi- 
tional stereotype for men. 
Voice—Pleasantness or quality. 


H. Abilities Affecting Relation with Others 


54- 


55: 


57: 


59- 
60. 


61. 


62. 


. Unselfishness—Patterns 


Social Intelligence—Ability to apply 
native insight to situations involving 
personal factors and insight into per- 
sonal factors as they affect situations. 

Social Skill—Skill in dealing with peo- 
ple so as to obtain a favorable reaction 
as the result of deliberate intention. 


General Characteristics Displayed in 
Relation to Others 


of behavior 
based upon an attitude toward the con- 
flicting interests of self and others. It 
does not include behavior motivated by 
a desire for approbation. 

Sympathetic . Nature—Habit of recog- 
nizing and being charitable toward the 
shortcomings of others, based upon an 
attitude toward weakness and _ short- 
comings. 


. Friendliness—Readiness with which an 


individual accords social acceptance 
easily to others. 

Sense of Humor. 

“Moral Courage”—Disposition on the 
part of an individual to be a leader for 
what he conceives to be ethically right 
in a conflict situation in spite of pos- 
sible unpleasantness. 


Characteristics Affecting Face-to-Face 
Relationships 


Forcefulness—Personality that domi- 
nates, that is ascendant, that is forceful. 
Self-Possession—Ability to display self- 
confidence in social situations, based 





63. 
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largely upon one’s attitude toward his 
own adequacy in those situations. 
Reserve—Habitual tendency to allow 
others to assume the dominant role in 
face-to-face situations. Unwillingness to 
become the center of attention. 


K. Characteristics Displayed in Relation to 


64. 


65. 


Others that Facilitate Understanding 


with Others 


Sincerity—Disposition on the part of an 
individual to be open and forthright in 
his relations with others. 
Tact—Habitual practice of knowing 
about and attending to those small 
details which in any given. situation 
will bring the least unfavorable reaction 
from other people. It does not include 
the sacrificing of principles. 


L. Characteristics Maintained in Relation to 
Others that Affect Joint Enterprises 


66. 


67. 


68. 


70. 


Impartiality—Habit of being impartial, 
and in matters of consequence not 
allowing mere personal acquaintance or 
prejudice to be an overruling factor. 
Leadership—Ability to command the co- 
operatior of others in enterprises upon 
the basis of their confidence in one’s 
judgment, courage, and integrity, plus 
the ability to actually direct enterprises 
with those qualities. 
Cooperativeness—Disposition to restrain 
for the welfare of the group impulses to 
command when circumstances seem to 
assign leadership to someone else or 
when authority resides in someone else, 
and to actively assist that person. 
Loyalty—Disposition to cooperate with 
a leader or one in authority because it 
is believed to be an obligation of 
subordinates. — 

Dependability—Habit of persisting in 
the effort to discharge responsibilities 
regardless of the interest of the task. 


M. Characteristics of the Reactions of 


71. 


72. 


Others to the Candidate 


Social Acceptance—The degree to which 
an individual’s whole life and conduct 
excites, in general, favorable reactions 
from other people. 
Estimableness—Quality of behavior 
such as receives the esteem and regard 
of other people. 


N. Characteristics in Relation to Life 
in General 


73. Social Point of View—Orientation in 
thinking, thought pattern. Disposition 
to criticize from the viewpoint of the 
welfare of society; inclination to in- 
clude social considerations when mak- 
ing decisions. 

74. Humility—Individual attitude toward 
the larger aspects of life. Attitude en- 
gendered by certain philosophical be- 
liefs as well as innate personal tend- 
encies and experience. 

75. Adaptability—Pattern of behavior 
which shows a willingness to attain ends 
by adjusting to circumstances rather 
than by trying to change the circum- 
stances as a means to the end. 

76. Service Point of View—Behavior 
actuated by the present conception of 
our culture that an individual has an 
obligation to serve his community. 

77. Experimental Attitude—Inclination on 
the part of an individual to explore the 
possibilities of novel methods, and a 
willingness to test in experience the 
desirability of untried values. 


O. Characteristics in Relation to Work 


78. Ambition—Long term desires for ad- 
vancement on the part of an individual 
usually translated into plans of action, 
which, however, are not always per: 
sisted in. 

79. Industriousness—Personal habit of ener- 
getic and persistent application to tasks 
at hand regardless of how discouraging 
preliminary results may be. 

80. Punctuality—Habit of strict adherence 
to time schedules and appointment 
hours. 

81. Carefulness—Behavior patterns marked 
by thoroughness, high standards for ac- 
complishment, and painstakingness. 

82. Thrift. 

83. Care in Financial Matters—Practice of 
meeting money obligations promptly 
without the necessity of pressure from 
others. 


P. Characteristics of the Personal Life 


84. Moral Character—Rigorous adherence 
to accepted ethical standards and incor- 
ruptibility in the face of temptation. 

85. Reverence. 
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TRAINING OR PREPARATION, IN 
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100. 
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Refinement—Education with respect to 
the customs and manners of the people 
and disposition to abide by accepted 
practices. It includes clean-mindedness. 


EXPERIENCE AND TRAINING 


. General estimate of both experience 


and training joined together. 

General estimate of both experience 
and training in the direction of extra- 
curricular activities. 


PHILOSOPHY OF EDUCATION 


General comment on the quality of 
the philosophy of education. (Such 
comment might seem worthless because 
of its highly subjective character; never- 
theless, a statement that a candidate 
had an excellent philosophy of educa- 
tion might impress some readers even 
though the writer of the recommenda- 
tion was unknown to the reader.) 


GENERAL 


General comment on the quality of the 
training or preparation, including com- 
ment on the general educational back- 
ground. 

Variety of training. 

Possession of capabilities and potentiali- 
ties for professional growth while in a 
position. 

Interest shown in professional prepara- 
tion and/or improvement as a student. 
Development shown as a student. Re- 
fers to general development rather than 
development solely in subject matter 
mastery. 


. Comment on a specific item of training 


or preparation not covered by other 
items of this section. 


. Preparation in the theory of teaching 


and education, or in professional edu- 
cation, or in professional courses. 


. Special training undergone. An exam- 


ple would be European study in the 
case of a candidate for a position to 
teach a foreign language. 

Training of special abilities and/or 
skills related to the teaching field in 
which the position was open. 

Interest in developing special abilities 
and/or skills related to the teaching 
field in which the position was open. . 
Training or preparation in the direc- 
tion of extra-curricular activities either 


101. 


from course work or participation as a 
student. 

Interest in extra-curricular activities as 
a student. 


KNOWLEDGE OF SUBJECT MATTER, SKILLS, 


102. 


103. 


104. 
105. 


106. 
107. 


108. 
109. 


110. 
111. 
112. 


113. 


114. 


115. 


116. 


117. 


AND ABILITIES 


General estimate of general scholarship 
or of work as a student. Not confined 
to one field or one professor. 

Interest on the part of a student in 
his studies or courses, in general. 
Wide learning of the candidate. 
Interest in a great variety of subjects 
and in wide learning in general. 
Knowledge of methods of teaching. 
Interest in educational problems from a 
theoretical viewpoint. 

Possession of demonstrated ability in 
extra-curricular activities. 

Research done by the candidate and/or 
ability to do research. 

Contributions to professional literature. 
Interest in research. 

Scholarship, skill, and ability in the 
teaching field in which the position 
was open. 


Note: When the position involved 
teaching subjects in more than one field, a 
reference to any one of the fields in which 
the position was open was counted. In the 
case of a position in science or general 
science, training in botany, zoology, geology, 
chemistry, physics,.or general science was 
counted as training in the field in which 
the ition was open. In the case of a 
position in social science, training in civics, 
citizenship, Problems of Democracy, his- 
tory, sociology, economics, or geography 
was counted as training in the field in 
which the position was open. 


Scholarship, skill, and ability in some 
other field than the one in which the 
position was open. 

Scholarship or work as a student in 
one course or under one professor in 
the teaching field in which the position 
was open. 

Scholarship or work as a student in one 
course or under one professor in some 
other field than the one in which the 
position was open. 

Quality of specific items of work, or 
specific learnings, abilities, or skills in 
the teaching field in which the position 
was open. 

Quality of specific items of work, or 
specific learnings, abilities, or skills in 
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some other field than the one in which 
the position was open. 

118. Candidate’s grades in the teaching field 
in which the position was open. 

119. Candidate’s grades in some other field 
than the one in which the position was 
open. 

120. Possession of special ability related to 
the work of the teaching field in which 
the position was open. 

121. Possession of special ability related to 
the work of some other field than the 
one in which the position was open. 

122. Interest in the teaching field in which 
the position was open. 

123. Interest in some other field than the 
one in which the position was open. 

124. Interest in the work of one class or 
under one professor in the teaching 
field in which the position was open. 

125. Interest in the work of one class or 
or under one professor in some other 
field than the one in which the position 
was open. 

126. Interest in the literature of the teaching 
field in which the position was open. 

127. Interest in the literature of some other 
field than the one in which the position 
was open. 


OCCUPATIONAL AND PROFESSIONAL ATTITUDES 
AND RELATIONSHIPS MAINTAINED 


128. General comment, such as professional 
interest. 

129. Interest in teaching as such. 

130. Interest in in-service improvement. 

131. Interest in the profession as an occu- 
pational group. 

132. Activity in professional organizations. 

133. Interest and willingness to work for the 
welfare of the institution by which 
employed. 

134. Interest in extra-curricular activities 
and willingness to help supervise and 
direct them. 

135. Relationships of cooperation and re- 
spect maintained in general. 

136. Relationships of cooperation and re- 
spect maintained with colleagues. 
137. Relationships of cooperation and re- 
spect maintained with supervisors. 
138. Relationships of cooperation and re- 

spect maintained with administrators. 

139. Relationships of cooperation and re- 
spect maintained with parents and 
toward their participation in the school 


program. 


ATTITUDES TOWARDS AND RELATIONSHIPS 
MAINTAINED WITH COMMUNITIES 


140. General comment on the attitude 
toward or the relationship maintained 
with the community, community inter- 
ests, or community affairs. 

141. Attitude toward or relationship main- 
tained with respect to the duties of a 
teacher as a citizen. 

142. Attitude toward or relationship main- 
tained with respect to the obligations 
of a teacher with regard to customs 
peculiar to the community. 

143. Attitude toward or relationship main- 
tained with respect to the expectations 
of the community with regard to 
church activity. 


SUCCESS OF EXPERIENCE AS A TEACHER, NOT 
REFERRING SPECIFICALLY TO CLASSROOM 
TEACHING 


144. General estimate of the experience or 
of the success of the candidate as a 
teacher. 

145. Wide variety of professional experience. 

146. Comment on amount or location of ex- 
perience which carries with it a defi- 
nitely favorable connotation. 

147. Quality of specific items of work or of 
special aspects of the experience. 

148. Special abilities related to the teaching 
field in which the position was open 
demonstrated while in a teaching posi- 
tion. 

149. Improvement shown while in a teach- 
ing position. 

150. Experience with extra-curricular and/or 
extra-position activities. 

151. Interest shown in extra-curricular and/ 
or extra-position activities. 

152. Experience in student guidance and 
counseling. 

153. Experience in the supervision of 
teachers. 

154. Experience in administration. 

155. Experience in teacher training. 

156. Experience in practice teaching as a 
student teacher. 


The. content, itself, of the following 
statements, indicative of the degree of 
success of experience, suggests the quality 
rating that is appropriate. The state- 
ments are listed in the Inventory because 
their content is classifiable. Normally a 
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statement was assigned the quality rat- 
ing given, but the rating was raised or 
lowered according to the modifying terms 
found and the context: 


157- 


164. 


105. 


166. 


107. 


168. 


173. 


Candidate received a contract for 
the next year. 


. School hopes that the candidate will 


remain on its staff. 


. Candidate may return to the posi- 


tion. 


. Candidate had been given a leave of 


absence. 


. School would be glad to have the 


candidate who left return. 


. School regrets or would regret losing 


the candidate, or does or would con- 
sider his leaving a loss. 


. Candidate who had. been: teaching 


in a temporary post would have 
been employed if a vacancy had 
developed. 

Candidate would be employed or 
continued in employment if there 
were a vacancy or if finances per- 
mitted. 

Candidate left voluntarily to accept 
another position. 

Candidate left voluntarily to study. 
Candidate left his position of his 
own accord. 

Statement that seems satisfactory as 
to why the candidate resigned from 
his position. 


. Suggestion of ways in which the 


candidate might improve. 


. Statement that arouses suspicion as 


to why the candidate resigned from 
his position. 


. Statement that the candidate taught 


one year in a position, with no 
further explanation. 


. Statement that the candidate had 


had experience when it is evident 
that the writer is in a position to 
evaluate that experience but refrains 
from doing so. 

Statement that the candidate failed, 
perhaps, with reasons being given. 


eo 


6 


7 


rECHNICAL SKILL IN TEACHING AS EXHIBITED 


IN THE CLASSROOM IRRESPECTIVE OF 


KNOWLEDGE OF SUBJECT MATTER 


174. General estimate of classroom teaching. 


175 


177. 
178. 
179. 


180. 
181. 


182. 


188. 


190. 


191. 
192. 


193. 


194. 


. Success in one aspect of classroom 
teaching. 

. Perspective with regard to the aims of 

education and/or the aims of the par- 

ticular course. taught. 

Influence on pupils. 

Understanding of young people. 

Teaching that indicates understanding 

of teaching methods and techniques. 

Personal interest in students. 

Habit of thorough and regular prepara- 

tion of classroom work. 

Ability to maintain control of a class 

(“discipline”). 


. Organization of teaching materials. 
. Apearance and/or manner before a 


class or with pupils. 


. Habit of attention to individual needs 


of pupils. 


. Willingness to do extra work to give 


assistance to individual pupils. 


. Habit of taking pains to motivate the 


work of pupils and/or the ability to do 
so. Includes getting a class to work 
diligently. 

Skill in exposition and presentation. 
Ability to translate knowledge of 
subject matter into terms useful in the 
classroom. 


. Ability to secure response from pupils 


and hold their interest. Includes ability 
to make interesting presentations. 

Skill in leading discussions and/or ask- 
ing questions. 

Acceptance of the teacher by the pupils. 
High standards of scholarship required 
of pupils and/or actually obtained. 
Habit of carefulness in routine matters 
connected with a position. 

Disposition to improve teaching skill 
and to keep up-to-date in methods and 
procedures. 


GENERAL ESTIMATE AND RECOMMENDATION 
AND / OR PREDICTION OF THE DEGREE OF 


195 


196. 


197. 


SUCCESS THAT WILL BE ACHIEVED 


. Recommendation for a position to 
teach in a secondary school, either pub- 
lic or private. 

Recommendation as a young or begin- 
ning teacher for a position in a second- 
ary school, either public or private. 
Recommendation for a teaching posi- 
tion, but without specific recommenda- 
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tion for a position in a secondary 
school. 

198. Recommendation as a young or begin- 
ning teacher for a position, but with- 
out specific recommendation for a posi- 
tion in a secondary school. 

199. Recommendation for a teaching posi- 
tion in other than a secondary school, 
if from the context it is judged that 
the writer was unaware that the candi- 
date would be interested in a secondary 
school position. 
Recommendation as a young or begin- 
ning teacher for a position in other 
than a secondary school, if from the 
context it is judged that the writer was 
unaware that the candidate would be 
interested in a secondary school posi- 
tion. 

201. Recommendation for a _ secondary 

school teaching position, but also for a 

position in administration or college 

teaching as well. 

Recommendation for a _ secondary 

school teaching position, and as a young 

or beginning candidate for a position 
in administration or college teaching. 

Recommendation for a _ secondary 

school position as a young or beginning 

teacher, and also for a position in ad- 

ministration or college teaching as a 

young or beginning candidate. 


200. 


202. 


203. 


The content, itself, of the following 
statements, indicative of the general es- 
timate made of the candidate, suggests 
the quality rating that is appropriate. 
The statements are listed in the Inven- 
tory because their content is classifiable. 
Normally a statement was assigned the 
quality rating given, but the rating was 
raised or lowered according to the modi- 
fying terms found and the context: 


204. Writer would be glad or very glad 
to employ the candidate. 2 
205. Writer has employed the candidate 
and that fact indicates how the can- 
didate is regarded. 2 
206. Candidate is qualified for super- 
vision. 2 
207. Candidate is qualified for teacher 
training. 2 


208. Candidate is also qualified for guid- hae a 


ance duties. 2 
Candidate is also qualified to assist 
in administration. 2 
Writer would be willing to employ 
the candidate. 3 
As an adjunct to the Inventory a List a 


of Equivalent Trait-Names and Phrases 
was constructed. During the construction i } 
of the section of the Inventory dealing : 
with personality traits, each trait-name et 
or phrase encountered was written on a pe 
separate card. Then by sorting cards, the 
trait-names and phrases were grouped 
into traits. One of the trait-names was 
chosen to represent the trait. All of the 
trait-names and phrases were then listed 
alphabetically with their Inventory 
equivalents. 


209. 


210. 


ee 








LIST OF EQUIVALENT TRAIT-NAMES 
AND PHRASES 


Note: A number following the trait-name or 
phrase indicates that it appears in the In- 
ventory as the item of the number given. 


A 


Ability—General Ability 
Ability to get along with people—Social 
Acceptance 
Ability to make decisions—42 
Accepts responsibility—Leadership 
Accomplished(ness)—General Ability 
Accuracy—Carefulness 
Activeness—F orcefulness 
Acumen—Intellectual Ability 
Acuteness—Intellectual Ability 
Adaptability—75 
Address—General Appearance 
Adjustable(ness)—Adaptability 
Administrative ability—Leadership 
Admirableness—Estimableness 
Affableness—Friendliness 
Aggressiveness—F orcefulness 
Agreeableness—Social Acceptance 
Alertness—Enthusiasm 


Aliveness—Enthusiasm 

All-round person—General Estimate ae 
Ambition—78 € 
Amiableness—Social Acceptance 

Amicableness—Friendliness Be 
Analytical(ness)—Intellectual Ability ; E 
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Appearance—General Appearance 
Application to work—Industriousness 
Appreciativeness—Humility 
Approachableness—Social Skill 
Ardentness—Enthusiasm 
Articulateness—Oral Expression of Thought 
Aspiration—Ambition 
Assiduousness—Industriousness 
Assurance—Self-Possession 
Astuteness—Intellectual Ability 
Athletic interests—Interest in Sports 
Attainments—General Ability 
Attentiveness—Carefulness 
Attitude toward criticism—Progressiveness 
Attitude toward moral and religious life— 
Moral Character 
Attitude toward self-improvement—Progres- 
siveness 
Attitude toward suggestions—Progressiveness 
Attractive personality—Social Acceptance 
Attractive personally—General Appearance 
Attractiveness—General Appearance 
Appreciation of music—47 
Appreciation of art—Esthetic Appreciation 
Artistic ability—Special Skill 


B 


Balance in thought—Emotional Balance 

Bearing—Appearance, Minor 

Beauty—Appearance, Minor 

Body build—Physique 

Breadth—Breadth of Interests 

Breadth of interests—44 

Breadth of reading—Breadth of Intellectual 
Interests 


Breadth of information—Breadth of Intel-. 


lectual Interests 
Breadth of intellectual interests—2g 
Breeding—Refinement 
Brightness—Intellectual Ability 
Brilliantness—Intellectual Ability 
Broad-mindedness—Critical Habits of 
Thought 
Build—Physique 
Buoyancy—Emotional Balance 
Businesslikeness—Carefulness 


C 


Calmness—Self-Possession 
Candidness—Sincerity 

Capability—General Ability 

Care in financial matters—83 

Care of property—Carefulness 
Carefulness—81 | 
Carriage of the body—Appearance, Minor, 


Character—Moral Character 

Charitableness—Unselfishness 

Charm—Social Skill 

Cheerfulness—Emotional Balance 

Chivalrousness—Refinement 

Clarity of expression—Expression of Thought 

Clean speech—Refinement 

Clean-cut(ness)—Moral Character 

Clear-cutness—Sincerity 

Cleverness—General Ability 

Common sense—Social Intelligence 

Companionableness—Friendliness 

Competency—General Ability _ 

Concentration—Industriousness 

Conduct—General Estimate 

Confidentness—Self-Con fidence 

Congeniality—Social Acceptance 

Conscientious worker—Industriousness 

Conscientiousness—Sincerity 

Considerateness—U nselfishness 

Consistent worker—Industriousness 

Conventionality—Refinement 

Conversation—Oral Expression of Thought 

Cooperativeness—68 

Cooperativeness without sacrificing the right 
to hold a_ personal opinion—Critical 
Habits of Thought 

Cordialness—Social Skill 

Courage of convictions—“Moral Courage” 

Courtesy—Refinement 

Creative imagination—Intellectual Ability 

Creativeness—Intellectual Ability 

Critical habits of thought—go 

Critical thinking—Critical Habits of Thought 

Cultural background—Refinement 

Current events, acquaintance with—Breadth 
of Intellectual Interests 


D 


Debts, payment of—Care in Financial Mat- 
ters 
Decisions, ability to make—42 
Delightfulness—Social Acceptance 
Democratic(ness)—Friendliness 
Dependability—7o 
Determinedness—Industriousness 
Dignity—Self-Possession 
Diligence—Industriousness 
Directness—Sincerity 
Diplomatic(ness)—Tact 
Discernment—Social Intelligence 
Discretion—Social Intelligence 
Discrimination—Social Intelligence 
Discrimination between ends and means— 
Perspective in Thinking 
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Discrimination between important and un- 
important elements—Perspective in Think- 
ing 

Discrimination between total plan and de- 
tails—Perspective in Thinking 

Dispassionateness—Objective Intelligence 

Disposition—Emotional Balance 

Dress—Appearance, Minor 

Drive—Industriousness 

Dutifulness—Loyalty 

Dynamic(ness)—Forcefulness 


E 


Eagerness—Enthusiasm 

Earnest worker—Industriousness 

Earnestness—Sincerity 

Easy to get along with—Social Acceptance 

Effective personality—Leadership 

Efficiency—General Ability 

Effort—Industriousness 

Emotional balance—38 ' 

Emotional stability—Emotional Balance 

Endeavor—Industriousness 

Energy—Industriousness 

Engagingness—Social Acceptance 

Enterprise—Industriousness 

Enthusiasm—39 

Eruditeness—Breadth of Intellectual Inter- 
ests 

Estimableness—72 

Esthetic appreciation—46 

Even temper—Emotional Balance 

Exactness—Carefulness 

Executive ability—Leadership 

Expansiveness—Friendliness 

Experimental attitude—77 

Experimental outlook—Experimental Aitti- 
tude 

Expresses himself well—Expression of 
Thought 

Expression of thought—32 


F 


Faces facts—Critical Habits of Thought 

Fair decisions—Impartiality 

Fair in personal relationships—Impartiality 

Fairmindedness—Impartiality 

Fairness in  thought—Critical Habits of 
Thought 

Fair play—Impartiality 

Faithful worker—Industriousness 

Faithfulness—Loyalty 

Fertility of imagination—Intellectual Ability 

Fidelity—Loyalty 

Fine person—Estimableness 


Fine character—Moral Character 

Flexibility—Adaptabilty 

Fluency—Oral Expression of Thought 

Force of character—Forcefulness 

Forcefulness—61 

Forcefulness of personality—61 

Foreign language spoken—Breadth of In- 
tellectual Interests, if it, does not classify 
elsewhere in the Inventory. 

Foresight—Social Intelligence 

Forgivingness—Sympathetic Nature 

Forthrightness—Sincerity 

Frankness—Sincerity 

Freedom from false pride or arrogance— 
Humility 

Freedom from prejudice—Critical Habits of 
Thought 

Freedom from rationalization—Critical Hab- 
its of Thought 

Friendliness—58 


G 


General ability—26 

Generousness—U nselfishness 
Geniality—Friendliness 

Gentlemen, a—Refinement 
Gentleness—Sympathetic Nature 
Genuineness—Sincerity 
Giftedness—General Ability 

Good associates—Moral Character 
Good-humoredness—Social Acceptance 
Good judgment—Social Intelligence 
Good-looking—General Appearance 
Good mixer—Social Skill 
Good-naturedness—Social Acceptance 
Good personality—General Estimate 
Good taste—Refinement 
Good-temperedness—Social Acceptance 
Good will—Unselfishness 
Graciousness—Social Skill 

Growth, personal—Growth in Personality 
Growth in personality—25 


H 


Habits, good—Moral Character 
Handsomeness—General Appearance 
Happiness—Emotional Balance 
Hard worker—Industriousness 
Healthiness—Physical Health 

Held in respect—Estimableness 
Helpfulness—Cooperativeness 
High-mindedness—Moral Character 
High-spiritedness—Enthusiasm 
Hobbies—Breadth of Interests 
Honesty—Moral Character 
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Humaneness—Sympathetic Nature 
Humility—74 


I 


Idealistic(ness)—Moral Character 

ideals—Moral Character 

Imagination—Intellectual Ability 

Imaginative power—Intellectual Ability 

Impartiality—66 

Impression the candidate makes personally— 
General Estimate 

Indefatigableness—Industriousness 

iIndependence—Self-Confidence 

Independence of thinking—Critical Habits 
of Thought © 

Industriousness—79 

Influential person—Estimableness 

Ingeniousness—Intellectual Ability 

Ingenuousness—Sincerity 

Initiative—Leadership 

Insight into problems involving personal 
factors—Social Intelligence 

Insight into problems not involving per- 
sonal factors—Intellectual Ability 

Integrity—Moral Character 

Intellectual ability—28 : 

Intellectual integrity—Critical Habits of 
Thought 

Intelligence—Intellectual Ability 

[.Q.—Intellectual Ability 

Intelligent person—Breadth of Intellectual 
Interests 

Interest in art—Esthetic Appreciation 

Interest in music—Appreciation of Music 

Interest in sports—45 

Interest in social questions—Social Point of 
View 

Interesting person—Social Skill 


J 


Jovialness—Sense of Humor 

Judgment in intellectual matters—Intellec- 
tual Ability 

Judgment in situations—Social Intelligence 

Justice—Impartiality 


K 


Keenness—Intellectual Ability 
Kindness—Unselfishness 
Kindheartedness—Sympathetic Nature 


L 
Lady, a—Refinement \ 
Ladylikeness—Refinement ) 
Leadership—67 


Learns rapidly—Intellectual Ability 

Leisure time activities, breadth of—Breadth 
of Interests 

Liberalness—Critical Habits of Thought 

Liberal, a—Social Point of View 

Lightheartedness—Enthusiasm 

Likeableness—Social Acceptance 

Liked by associates—Social Acceptance 

Loyalty—69 

Lovableness—Social Acceptance 

Lucid expression—Expression of Thought 


M 


Magnanimousness—U nselfishness 

Magnetism, personal—Social Skill 

Manliness—52 

Manners—Refinement 

Maturity—37 

Membership in clubs—Social Acceptance 

Mental peculiarities, freedom from—Emo- 
tional Balance 

Mind, bright or alert—Intellectual Ability 

Modesty—Reserve 

Moral Character—84 

“Moral Courage”—60 

Morale—Enthusiasm 

Manner—Social Skill 

Musical ability—Special Skill 

Music, appreciation of—47 


N 


Native ability—Intellectual Ability 
Native endowment—General Ability 
Neatness—Appearance, Minor 
Niceness—Refinement 
Normalness—Emotional Balance 


O 


Objective intelligence—41 

Objectivity—Emotional Balance 

Objectivity in thinking—Objective Intelli- 
gence 

Obedience—Loyalty 

Open-mindedness—Critical Habits of 
Thought 

Optimism—Enthusiasm 

Oral expression of thought—33 

Orderliness—Carefulness 

Organizing ability—Leadership 

Originality—Intellectual Ability 

Outgoingness—Friendliness 

Outspokenness—Sincerity 


P 
Painstakingness—Carefulness 
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Patience—Sympathetic Nature 

Peculiarities, freedom from—Emotional Bal- 
ance 

Periodical reading—Breadth of Intellectual 
Interests 

Perseverance—Industriousness 

Persistence—Industriousness 

Personableness—Social Acceptance 

Personal qualities—General Estimate 

Personal qualifications—General Estimate 

Personality—General Estimate 

Personality, attractive—Social Acceptance 

Perspective in thinking—g} 

Physical health—35 

Physique—36 

Pleasant manner—Social Acceptance 

Pleasant personality—Social Acceptance 

Pleasant voice—53 

Pleasantness—Friendliness 

Pleasing personality—Social Acceptance 

Pluck—Industriousness 

Poise—Self-Possession 

Politeness—Refinement 

Popularity—Social Acceptance 

Positiveness—Forcefulness 

Practicalness—General Ability 

Preciseness—Carefulness 

Prepossessingness—Forcefulness 

Progressiveness—43 

Progressive in outlook—Social Point of View 

Progressive in thinking—Critical Habits of 
Thought 

Promptness—Punctuality 

Prudence—Social Intelligence 

Public speaking—Oral Expression of Thought 

Public spirit—Service Point of View 

Punctuality—80 

Purposefulness—Industriousness 


Q 

Questions, character of the kind the candi- 
daté asks—Intellectual Ability 

Qualities of character that will enable the 
candidate to succeed in teaching—General 
Estimate 

Quickness—Intellectual Ability 

Quietness—Reserve 


R 


Racial tolerance—Sympathetic Nature 

Realistic thinking—Critical Habits of 
Thought 

Reasonableness—Emotional Balance 

Recommended as a person—General Estimate 

Recreational interests—Breadth of Interests 


Refinement—86 
Reliability—Dependability 
Religious interests—Reverence 
Religious tolerance—Sympathetic Nature 
Reputation, good—Moral Character 
Reserve—63 
Responsibleness—Dependability 
Responsiveness—Friendliness 
Resourcefulness—General Ability 
Reticence—Reserve 

Reverence—85, 


S 


Sagacity—Social Intelligence 
Saneness—Emotional Balance 
Sanguineness—Enthusiasm 
Sedulousness—Industriousness 
Self-confidence—40 
Self-control—Self-Possession 
Self-disciplined(ness)—Self-Possession 
Self-made person—General Ability 
Self-possession—62 
Self-reliance—Self-Con fidence 
Self-restraint—Self-Possession 
Sense of propriety—Refinement 
Sensibleness—Emotional Balance 
Sensitiveness (responsiveness to human val- 
ues)— Tact 
Seriousness—Industriousness 
Serenity—Emotional Balance 
Simplicity—Humility 
Sincerity—64 
Skillfulness—General Ability 
Smartness—Intellectual Ability 
Sociableness—Friendliness 
Social point of view—73 
Social standing—Refinement 
Society, background in—Refinement 
Society, upbringing in—Refinement 
Special skill or ability, general estimate of-- 
Special Skill 
Spirit—Enthusiasm 
Stableness—Emotional Balance 


‘Steadiness—Emotional Balance 


Straightforwardness—Sincerity 
Straight thinking—Intellectual Ability 
Strong personality—Forcefulness 
Sweetness—Social Acceptance 
Sympathetic Nature—57 


' Sense of humor—59 


Service point of view—76 

Social acceptance—71 

Social intelligence—54 

Social skill—55 

Skill in dealing with people—Social Skill 
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T 
Tact—65 
lalented(ness)—General Ability 
l eachableness—Adaptability 
lemperateness—Refinement 
( horoughness—Carefulness . 
lhoughtfulness—Unselfishness 
Chrift—82 
lolerance—Critical Habits of Thought 
folerance of people—Sympathetic Nature 
| rustworthiness—Dependability 
(ruthfulness—Moral Character 
lravel—48 


U 
U nassumingness—Humility 
‘nderstanding nature—Sympathetic Nature 
nderstanding of ideas—Intellectual Ability 
‘nderstanding of situations—Social Intelli- 
gence 
Jnprejudiced(ness)—Critical Habits of 
Thought 
Unselfishness—56 
U p-and-comingness—Progressiveness 
Use of English—Expression of Thought 


Vv 
Versatileness—General Ability 
Vigorousness—Forcefulness 
Vitality—Forcefulness 
Vivaciousness—Enthusiasm 

O1ce—53 


WwW 


Well-bred—Refinement 

Well-read—Breadth of Intellectual Interests 

/Vholesomeness—Moral Character 

Will speak up for what is considered to be 
right—““Moral Courage” 

\Villing worker—Industriousness 

Winsomeness—Social Acceptance 

Wisdom—Social Intelligence 

Vorks well with people—Cooperativeness 

Written expression of thought—34 


¥ 


roung man—5l 
young woman—5l 


Z 
/Zealousness—Enthusiasm 


In classifying comments with this list 
; a guide, some discretion had to be 
xercised, Alertness, for example, was 

) 


frequently used in a context which 
clearly indicated that general mental 
alertness was referred to rather than gen- 
eral physical attentiveness, and accord- 
ingly the comment. was classified under 
General Ability. Address may refer to 
bearing and conduct while approaching 
another, or to manner of speaking to 
another. Accordingly whether a com- 
ment upon address was classified under 
General Appearance or Expression of 
Thought depended upon the context. If 
the context did not suggest a decision, 
the classification listed was made. 

Most of the items of the Inventory are 
either favorable or neutral. There are 
some unfavorable items, but these are 
given specific quality ratings in the In- 
ventory (items numbered 169, 170, 171, 
172, 173). The direct criticism of candi- 
dates by the use of a word of definitely 
unfavorable denotation was most rare, 
occurring not more than five or six times 
in the total of 1035 different recommen- 
dations, if that often. Therefore it was 
unnecessary to list the antonyms of the 
trait-names appearing in the Inventory. 

The one item that required special 
treatment was labelled Reserve (No. 63) 
and defined as “Habitual tendency to 
allow others to assume the dominant role 
in face-to-face situations. Unwillingness 
to become the center of attention.” 
Sometimes when this trait is mentioned, 
the connotation, at least, is unfavorable. 
However, it was the judgment of an ex- 
ecutive with placement experience, one 
of the co-sponsors of this study, that not 
all references to reserve were to be taken 
as being unfavorable. In the analysis of 
the recommendations, every reference to 
reserve was taken by the author to this 
executive for consultation, and together 
the quality rating was decided upon in 
the light of the context in which the 
trait was mentioned. 
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The rating of statements for quality 
and the Quality Rating Scale are based 
upon the proposition that the favorable- 
ness of the modifying terms used by the 
writer of the recommendation is an index 
of his enthusiasm in recommending the 
candidate and therefore of the quality 
of the recommendation. The scale was 
prepared in the following way. A long 
list of adjectives and adverbs was com- 
piled by studying a large number of rec- 
ommendations’ and augmented by search 
in the dictionary and Roget’s Thesaurus 
for synonyms and antonyms. The adjec- 
tives and adverbs of the final list were 
classified by the author into seven cate- 
gories which were chosen to represent 
seven degrees of enthusiasm in recommen- 
dation of a candidate or seven degrees of 
favorableness-unfavorableness in modify- 
ing terms. The seven categories with the 
adjectives chosen to define them are as 
follows: 


Rating Term 

best 

superior 

above average 
average 
mediocre 
below average 
inferior 


“I GOt hm OO hb 


QUALITY RATING SCALE 


Nore: The ratings of the terms marked with 
an asterisk represent group decisions by the 
sponsors of the study and the author. 


Rating of 1 
absolutely "distinctly 
abundant distinguished 
all entirely 
always especially 
amazing exceedingly 
among the best excellent 
astonishing exceptional 
best extraordinary 
brilliant extremely 
commanding faultless 
complete grade of A 
conspicuous greatest 
consummate highest 
decided imposing 


impressive remarkable 
incomparable signal 
incredible singular 
inimitable striking 
intensely strongest 
largest superlatively 
matchless surpassing 
more than any other top 10 per cent 
most unapproached 
most with a modifier unequaled 

rated 2 or 3 unique 
notable unlimited 
noteworthy unparalleled 
one of the best unqualifiedly 
paramount unquestionably 
particularly unrivaled 
peculiarly unsurpassed 
peerless vast 
perfect very outstanding 
preeminently very splendid 
profound very superior 
pronounced without reservations 
rare 

Rating of 2 

ample superior 
altogether thoroughly 


altogether with a 
modifier rated 2 
or 3 

admirable 

comprehensive 

considerable 

considerably above 
average 

expert 

extensive 

first-class 

first-rate 

grade of B+ 

in all situations 

in every way 

noticeable 

outstanding* 

plentiful 

proficient 

quite with a modi- 
fier rated 2 or 3 

rather with a modi- 
fier rated 2 

reputable 

solid 

splendid 


thoroughly with a 
modifier rated 2 
or 3 

top 25 per cent 

uncommon 

unordinary 

very 

very with a modifier 
rated 2 or 3 

Exceptions: 
very outstand- 
ing 

very splendid 
very superior 

very able 

very capable 

very competent 

very fine* 

very great* 

very high 

very strong 

very successful 

well with a modifier 
rated 2 or 3 

without hesitation 


Rating of 3 


able 


above average 
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above par praiseworthy 
altogether with a _ quite 

modifier rated 4 quite good* 
appreciable quite well* 
asset quite with a modi- 


better than average fier rated 4 
apable rather with a modi- 
creditable fier rated 3 
commendable real* 
competent rich 
deep skillful 
lesirable sound 
effectively strong 
efficient substantial 
favorable successful 
fine* surprising 
full thoroughly with a 
erade of B modifier rated 4 
ereat amount thoroughly good 
great deal top 33 per cent 
high unusual 
high class valuable 
indeed very good* 
keen very well* 
large very with a modifier 
liberal rated 4 
marked well above the aver- 
nodel age 
more than average weighty 
much well fitted 
not wanting well qualified* 
over par wide 
plenty worthy 
Rating of 4 
icceptable qualified 
idequate quite good* 
ll right quite well* 
at par rather 
iverage rather good* 
omparatively rather well* 
enough rather with a modi- 
equalled fier rated 4 
fit satisfactory 
sood* some 
good enough somewhat 
middling sufficient 
moderate usual 
modest well* 
passable well enough 
Rating of 5 
common fairly good* 


fair 


I rly 


fairly well* 
fairly with a modi- 


fier rated 4 or 5 





pretty good* 


incomplete pretty well* 

indefective pretty with a modi- 

indeficient fier rated 4 or 5 

mediocre second-class 

not especially second-rate 

not particularly tolerable 

not so good undistinguished 

ordinary unimposing 

pretty unimpressive 
Rating of 6 

below average not so 

below par objectionable 

excessively over 

faulty overly 

few short of 

handicapped slight 

inadequate small 

lacks third-class 

less than average third-rate 

limited under par 

little wanting in 

low weak 

low class worse 

not as worse than average 


not enough 





Rating of 7 
defective no 
deficient not 
dull poor 
failing scant 
feeble -shallow 
inappreciable superficial 
incompetent thin 
indifferent too little 
inferior un 
insufficient unnoticeable 
lowest unsound 
meager unsatisfactory - 
minus worst 


The easiest type of statement to 
handle is one in which the word or 
phrase identifying the trait or qualifica- 
tion is completely neutral and its use 
does not imply degree of evaluation. For 
instance, “He is a teacher” cannot be 
rated. It is a statement which may or 
may not be true, but it cannot be 
evaluated for degree to which the trait is 
possessed and therefore cannot be eval- 
uated for quality as defined in this study. 
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But the statement “He is a good teacher” 
can be evaluated. A modifying term has 
been added which implies the degree to 
which the trait is possessed by the candi- 
date in the opinion of the writer of the 
recommendation. This statement would 
be rated 4 because good is rated 4 in the 
Quality Rating Scale. “He is a strong 
teacher” would be rated 2, and so on. 
The quality rating is simply based upon 
the modifying term. In such statements 
the modifying terms themselves carry all 
of the implication of the degree to which 
the trait or qualification is possessed. 

Some evaluations, however, are made 
without the use of terms modifying the 
word or phrase identifying the trait or 
qualification imputed to the candidate. 
Such statements cannot be referred to 
the Quality Rating Scale. For example, 
“He is enthusiastic.” The adjective en- 
thusiastic is not qualified and there are 
no modifiers to rate. The adjective serves 
to identify the trait possessed by the 
candidate, but it also indicates the de- 
gree to which the trait is possessed. Such 
statements were given a quality rating 
of 3. It was assumed that because the 
trait had been singled out for mention 
in the recommendation it was possessed 
to above-average degree. 

If the adjective serving to define the 
trait commented upon had been quali- 
fied by a favorable adverb, the rating 
would have been raised to a 2. For exam- 
ple, “He is very enthusiastic.” The rule 
used was that the addition of the favor- 
able modifier raises the rating one point. 
Another favorable modifier would raise 
the rating another point. Thus, “He is 
very markedly enthusiastic’ would be 
rated 1. Unfavorable modifiers would 
reduce the rating in a like manner. 

If a single adverb from the list rated 1 
had modified the adjective, the quality 
rating would have been 1, For example, 


“He is particularly enthusiastic.” And 
the same principle would hold for modi- 
fiers from the list rated 7. : 

In case the trait possessed by the candi- 
date was identified by a noun, a rating of 
3 was given when the noun indicated 
that the trait was singled out for men- 
tion by the writer. Thus, “He has en- 
thusiasm” would be rated 3. “He has 
much enthusiasm” would be rated 2 be- 
cause the original rating of 3 would be 
increased one point for the additional 
favorable modifier. In all of these cases 
the word or phrase identifying the trait 
implies that the trait is possessed to some 
degree. 

In other cases the rating was based 
strictly upon the modifier. “His en- 
thusiasm is marked” and “He is strong 
in enthusiasm” would both be rated 3 
because marked and strong are in the 
list rated 3. These statements were not 
rated 2 just because a favorable modifier 
is found with a trait-name. In the fore- 
going cases, the modifiers carry all the 
qualification expressed; the trait-name is 
used solely to identify the trait being 
referred to. The distinction is between 
the statement in which the trait-name 
serves only to identify the trait about 
which the statement of degree is made, 
and the statement in which the trait 
name is used in such a way that it signi- 
fies degree as well as identifying the trait. 
In the latter case the mere use of the word 
or phrase identifying the trait implies 


evaluation of the candidate. 


Certain modifying terms have the 
effect of reinforcing the quality designa- 
tion in such a way as to raise the quality 
rating. “In every way,” “in all situa- 
tions,” “indeed,” and “without hesita- 
tion” are examples. “He is cooperative 
in all situations” would be rated 2 
instead of 3, the rule being to raise the 
rating one point. Underscoring of the 
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word indicating quality was interpreted 
in the same way. 

In general, whenever the rating was 
based strictly upon the modifying terms 
and a number of favorable modifying 
terms were found in a series, the rating 
was based upon the following scheme: 


Modifiers Rating 
Class Class 
1 1 1 
1 2 1 
l 3 1 
2 4 3 
2 2 2 
2 3 2 
- 4 3 
8 3 2 
3 4 3 
4 4 4 


Certain exceptions as well as certain of 
he more common combinations are to 
be found in the lists of the Quality Rat- 
ing Scale. 

Item No. 51 of the Inventory, a state- 
ment that the candidate is a young man 
or young woman, was not rated but was 
simply checked as being present. If any 
estimate of the candidate was qualified 
by such a phrase as “for a young per- 
son,” the quality rating was reduced one 
point. ‘Thus, “For a young man he had 
‘ecided ability in leadership” would be 
ated 2 rather than 1.8 

The rating for quality of statements 

lassified under the section of the In- 
entory, General Estimate and Recom- 
nendation and/or Prediction of the De- 
gree of Success That Will Be Achieved, 
vas accomplished with a scale adapted 


*Record was also made of information use- 
‘ul for weighing purposes but not required for 
the investigation reported here. If there was 
‘efinite indication of the degree of confidence 
f the writer in his evaluations, it was recorded. 

the evaluation of the writer seemed to be 
ased upon observation of the candidate in a 
umber of positions in the field. that fact was 
I corded. 


\ 


i 
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from one proposed by Weidemann 
(20, p. go). The following categories 
were adapted from Weidemann’s scale 
and assigned the quality ratings indi- 
cated: 


1. Most heartily recommend and believe 
this teacher to be one of the best 
equipped of this year’s teachers, or one 
of the best teachers of my staff, or one 
of the best teachers of my acquaintance. 

2. Am glad to give this teacher my cordial 

recommendation. 

. Willingly recommend this teacher. 

. Believe this teacher may be safely rec- 

ommended. 

5. Feel unable to express myself either fa- 
forably or unfavorably with respect to 
this teacher. 

6. Am in doubt as to whether this teacher 
should be recommended. 

7. Cannot recommend this teacher. 


He OO 


At the time the job-situations and 
candidates were initially selected, sets of 
credentials containing the recommenda- 
tions, there being one set for each candi- 
date, were alphabetized by name of 
candidate for each job-situation. Then 
the job-situation sets were put in order 
by subject matter field. This was done 
in such a way as to leave no clue as to 
which candidate was the successful one. 
During two years which had elapsed 
from the time the credentials were taken 
from the files and the job-situations were 
originally studied to the time when the 
process of analyzing the recommenda- 
tions was begun, the author had com- 
pletely forgotten the identity of the suc- 
cessful candidates. 

The author studied each recommenda- 
tion, analyzing it into a series of state- 
ments, and recorded the data in two 
parallel columns as in the Illustration of 
Recommendation Analysis. All of the 
recommendations for all of the candi- 
dates of a given job-situation were 
analyzed at the same time, candidate by 
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candidate. The data were recorded by 
recommendation, by candidate, by job- 
situation, the trait or qualification com- 
mented upon being recorded in one 
column with the Inventory number, and 
the quality rating being recorded in a 
second column opposite the trait num- 
ber. 

If a number of statements in succes- 
sion referred to the same trait, the 
quality rating which seemed to represent 
the impression made by the whole series 
of statements was recorded as well as 
individual quality ratings for each state- 
ment so that a single rating representa- 
tive of the whole series might be em- 
ployed in obtaining a score for the 
recommendation, and the Inventory 
number was recorded with ditto marks. 
When, however, the trait is one which 
might cover a number of instances of 
the qualification, the Inventory number 
was repeated, and the quality of each 
statement was given a separate rating. 
An example would be the specification 
of a number of different extra-curricular 
activities. 

Comments of the writer regarding the 
importance of traits or qualifications 
were disregarded. For example, no atten- 
tion was paid to the last clause in this 
statement, “She has a dynamic and like- 
able personality, which is salient.” 

For statements of non-ratable informa- 


tion, again the Inventory number was 
recorded in the first column, but a dash 
was recorded in the second column to 


indicate that the statement did not re- 
ceive a quality rating but was to be con- 
sidered as having contributed to the 
length of the recommendation. 

The process of analyzing the recom- 
mendations depends in the last analysis 
upon the judgment of the research 
worker and is admittedly subjective. The 
unreliability inherent in the judgment 
of a single individual might have been 
overcome by increasing the number of 
subjective judgments and then taking 
the concensus, but it appeared impos- 
sible to find other judges who could go 
through the process the author did be- 
fore analyzing the recommendations. 
The whole process of reading hundreds 
of recommendations with definite prob- 
lems of the research in mind, of con- 
structing the Inventory, of classifying 
modifying terms in the Quality Rating 
Scale, and of classifying synonyms and 
antonyms of personality traits seemed to 
be a necessary preparation for the final 
process of analyzing the recommenda- 
tions themselves. This preliminary proc- 
ess required months of concentrated 
effort. It did not seem feasible for others 
to go through the same process. The 
alternative was to employ every possible 
means to make the work of the author 
as objective and consistent as possible. 
The Inventory, the Quality Rating 
Scale, and the List of Equivalent Trait- 
Names and Phrases helped in that 
process, and every precaution was taken 
to insure care in the work. 
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ILLUSTRATION OF RECOMMENDATION ANALYSIS 


The Recommendation: 
(S7 K5) 
2/19/36 


Mr. K has been connected with the local schools for the past five 
years as a science instructor and now as Principal of our senior 
High School, a promotion he won by his services within our school 
system. | 


Mr. K has an excellent personality, pleasing to meet, and one who 
grows as you become better acquainted with him. He possesses an 
excellent character, high ideals, correct attitudes and a practical 
viewpoint. He is neat in his personal appearance. 


Mr. K is interested in community affairs, loyal to the institution 
that he is connected with, and his associates. He has a very good 
influence over the student body and is excellent in helping to solve 
their problems. He has social sanity and a keen sense of values. 
He has his work well organized, makes excellent preparations, ex- 
ecutes his plans efficiently and secures good results. 


While I have been acquainted with Mr. K only for the past year, 
yet my close association with him during this time proves to me 
his excellent work and insight into his future growth and possi- 
bilities. I am more than pleased to recommend Mr. K. 


Signed: H. F. N. 
Superintendent of Schools 
ky ‘ES 











The Analysis: 
Statement Inventory Quality 
Item Rating 
1. Date of the recommendation 
2. Mr. K has been connected with the local schools. 16 _— 
3. Mr. K has been connected with this system for the past five years. 15 — 
4. Mr. K has served as science instructor and now as Principal of our senior 
High School. 17 i 
5. The latter position was a promotion he won by services within our school 
system. 144 3 
6. Mr. K has an excellent personality. 24 1 
7. Mr. K is pleasing to meet. 71 3 
8. Mr. K is one who grows as you become better acquainted with him. 72 3 
g. He possesses an excellent character. 84 i* 
10. He possesses high ideals. “ 3 
11. He possesses correct attitudes. ss 3 
12. He possesses a practical viewpoint. —_ — 
13. He is neat in his personal appearance, 50 3 
14. Mr. K is interested in community affairs. 140 3 
15. Mr. K is loyal to the institution that he is connected with. 133 3 
16. Mr. K is loyal to his associates. 135 3 
17. He has a very good influence over the student body. 177 3 
18. He is excellent in helping to solve their problems. 178 1 
19. He has social sanity. 10 — 
20. He has a keen sense of values. 31 3 
21. He has his work well organized. 183 4 
22. He makes excellent preparations. 181 1 
23. He executes his plans efficiently. 193 3 
24. He secures good results. 174 4 
25. While I have been associated with Mr. K only for the past year, yet my close 
association with him during that time — — 
26. Proves to me his excellent work. 144 1 
27. My association with Mr. K gives me insight into his future growth and 
possibilities. 92 3 
28. I am more than pleased to recommend Mr. K. 195 2 





Sum of quality ratings — 50. 
Mean quality rating- (50 + 21) = 2.381. 


* The quality rating indicated was judged to represent the impression made by the series of three 
statements about the same Inventory item. 
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CHAPTER 4 


Factors AFFECTING NOMINATION AND PLACEMENT SUCCESS 


HETHER a candidate is successful in 
W obtaining a teaching position 
through the agency of a placement office 
depends upon his surviving two processes 
of selection; one is nomination by the 
placement executive handling the in- 
quiry, and the other is appointment by 
the employer. In both cases the decision 
rests upon a variety of factors. Three 
principal kinds of information regarding 
the candidate are possessed by the per- 
sons responsible for selection and form 
the basis for their decisions. These are 
é impressions from personal contact, fac- 
tual information, and evaluations of the 
candidate made by others. Each of these 
in turn is obtained principally from a 
inique source. Impressions of the candi- 
date are usually gained from interviews, 
this being particularly so for the em- 
ployer. Factual information is frequently 
found in the recommendations but is 
primarily obtained from_ biographical 
forms which usually are included in a 
set of credentials. Evaluations of the 
candidate made by others are found 
principally in the recommendations. 
The pertinent procedures of the Office 
of Placement Service of Teachers College 
ire as follows. In response to a request 
for nominations, sets of credentials are 
either mailed or given to the employer 
for examination. A set of credentials con- 
sists of a cover page identifying the can- 
lidate, a photograph, six pages of 
biographical information, and copies of 
he recommendations the candidate has 
on file. The candidate is; allowed to 
choose from whom he will request recom- 
mendations and these are secured with 
ihe form referred to in Chapter 3. The 
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biographical information is solicited 
through a series of questions which the 
candidate answers as he chooses. 

The following information about the 
candidate and his qualifications is 
typically available from the credentials. 


Cover Page 


1. Name of the candidate. 

2. Photograph showing the head and shoul- 
ders. 

Present address. 

Present position. 

Date the credentials were compiled. 


as i 2c 


Personal (Page 1) 


1. Date the candidate filled the biographical 

forms. 

Name of the candidate. 

Permanent address. 

Year of birth. 

Marital status. 

Number of children. 

Church. 

Height. 

Weight. 

Name of high school from which gradu- 

ated. 

. Location of high school from which grad- 
uated, 

. Date of high school graduation. 

. Record of attendance at higher institu- 
tions of learning including Teachers Col- 
lege. 

a. Name of institution. 
b. Dates of attendance. 
c. Degrees received, 

14. Degree for which the applicant is at 

present a candidate. 

15. Approximate date on which this degree 

is to be conferred. 

16. Honorary fraternities. 

17. Foreign languages spoken. 

18. Types of positions the candidate is pre- 

pared to accept. 
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Undergraduate Courses (Page 2) 
1. Name of the candidate. 
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. List of undergraduate courses by years. 
. Evaluation of undergraduate experience 
by the candidate. (The question reads: 
What specific abilities or attitudes did 
you get in college which, in your opinion, 
should make you of value to an em- 
ployer?) 


Graduate Courses (Page 3) 


. Name of the candidate. 

. List of graduate courses taken at Teach- 
ers College and Columbia University. 
a. Name of the instructor. 

b. Title of the course (not catalog num- 
ber). 

. Graduate courses taken elsewhere. 

a. Name of the institution. 

b. Name of the instructor. 

c. Title of the course. 


Experience (Page 4) 


. Name of the candidate, 

. Experience. 

a. Dates. 

b. Name of the institution. 

c. Location-of the institution. 

d. Type of work. 

. Comment. (The instructions read: Men- 
tion any outstanding features of your 
training and experience not covered 
above or under questions on Page 5.) 


Extra-Curricular Activities (Page 5) 


. Name of the candidate. 

. List of activities in which the candidate 
can participate. 

. List of activities which the candidate can 
direct. 

. Comment. (The instructions read: Write 
specifically of your interest or experi- 
ence in these or other activities.) 


Candidate’s Page (Page 6) 


. Name of the candidate. 

. The instructions read: This is your page. 
On it you may write whatever you please 
to serve whatever purpose you may have. 


The Recommendations 


Answers to additional questions, 


though not directly provided by this in- 
formation, can frequently be inferred, 
or with the aid of the information can 


be secured from other sources. The fol- 
lowing items are of this type: 


1. 


o> 


10. 
11. 
12. 
13. 
14. 
15. 
16. 


17. 
18. 


19. 
20. 
21. 
22. 
23. 
24. 


Geographical or regional location of early 
environment. 

Sociological character of early environ- 
ment. 

Urban or rural. 


. Nationality as judged from the name, 


and, perhaps, from the location of the 
school where secondary education was ob- 
tained. 


. Race as judged from the photograph. 
. Type of secondary school attended. 


a. Public or private. 
b. Public or parochial. 


. Type of collegiate education. 


a. Teachers college, liberal arts college, 
or university. 
b. Coeducational or otherwise. 


- Record of persistence on the part of the 


candidate in the face of apparent diffi- 
culties in order to obtain an education. 


. Specific listing of the type of the posi- 


tion open as one the candidate is pre- 
pared to accept. 

Number of different types of positions 
the candidate is prepared to accept. 
Variety of positions for which the candi- 
date considers himself qualified. 

Subject matter listed first by the candi- 
date. 

Judging from the list of courses, the can- 
didate’s undergraduate major. 

Judging from the list of courses, the can- 
didate’s undergraduate minors. 

Judging from the list of courses, the can- 
didate’s graduate major. 

Judging from the list of courses, the can- 
didate’s graduate minors. 

Location of the majority of the courses 
taken. 

In the subject matter field or in educa- 
tion. 

Number of years of teaching experience. 
Number of situations in which the teach- 
ing experience has been gained. 

Grade or age groups taught. 

Subjects taught. 

Experience in guidance and counseling. 
Experience in supervision. 

Experience in administration. 

Types of schools in which the experience 
has been gained. 

a. Public or private. 
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b. Coeducational or otherwise. 

c. Urban or rural. 

d. “Progressive” or otherwise. 

e. Elementary, secondary, or collegiate. 
Variety and amount of summer teaching 
experience. 

26. Variety, amount, and, perhaps, the value 
of experience other than professional 
work in education. 

Present state of employment. 
Versatility in extra-curricular activities. 
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The sections for comment by the can- 
didate on pages 2, 4, and 5 together with 
the Candidate’s Page frequently give 
clues respecting the following items of 
information: 


1. The age group in which he is most inter- 
ested. 

2. The seriousness of his interest in the vo- 
cation of teaching and the motives that 
led to his selection of the profession. 

3. The degree of his personal interest in 
boys and girls. 

4. Something of his philosophy of educa- 
tion. 

5. His skill and interest in different extra- 
curricular activities. 

6. The hobbies and avocations he most en- 
joys. 

7. The entertainment and recreations he 
most enjoys. 

8. The variety and value of his work ex- 
perience. 

g. The variety and value of his travel ex- 
perience. 

10. The values in his own education he con- 
siders most significant. 

11. Something of the development of his 
own personality. 

12. Perhaps something of his orientation 
with respect to social problems. 

13. Something of his philosophy of life. 
When considering candidates an em- 

ployer generally has in mind certain 

particular requirements, two aspects of 

which are important. A requirement 

may be specified in the inquiry or cor- 

respondence, or not specified. And a re- 

quirement may be reasonable or un- 

reasonable. A requirement specified in 

ihe inquiry, which will be called ,a 


| 


specification, may be either reasonable 
or unreasonable; and an unspecified re- 
quirement, likewise, may be either 
reasonable or unreasonable. For obvious 
reasons there is a tendency for specifica- 
tions to be reasonable and for unreason- 
able requirements to be unspecified. Re- 
quirements that are taken for granted 
may seem to form a special class, but 
because of the way in which they operate 
should be classified with specifications. 
For example, an employer may fail to 
mention a sex requirement for a position 
in a girl’s boarding school, but experi- 
enced placement executives will assume 
that the candidate must be a woman. 
Whether a given requirement in a given 
job-situation is reasonable or unreason- 
able is a matter of opinion, and a de- 
tailed basis for distinction between the 
two kinds of requirements is not neces- 
sary for this study. It is enough to have 
it understood that these requirements 
run the gamut from stipulations that 
nearly everyone would agree are reason- 
able to those that nearly everyone would 
agree are matters of prejudice. It is, of 
course, impossible to demonstrate that 
an unspecified requirement has actually 
operated in a given job-situation; but, 
nevertheless, one cannot go to the op- 
posite position and assume that because 
requirements have not been specified, 
they have not operated. With respect to 
unspecified requirements, the research 
worker is forced to deal with possibili- 
ties. 
Some inquiries give definite informa- 
tion about the character of the position 
to be filled and about the qualifications 
desired in the candidate—as, for instance, 
an inquiry calling for a man who had 
been graduated by one of three men’s 
colleges of a certain state. Other in- 
quiries have some specified require- 
ments, but not many and not very nar- 
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row requirements. In fact, frequently the 
inquiry itself refers to the desired quali- 
fications as preferences rather than re- 
quirements. The more reasonable and 
more usual the qualification desired, the 
more likely the employer is to look upon 
it as a requirement rather than a prefer- 
ence. A surprisingly large number of 
inquiries are received with a bare mini- 
mum in the way of specifications and 
description of the position. Even though 
it is one of the most important items of 
information, not all inquiries mention 
the salary or salary range and other re- 
wards attached to the position. Some- 
times specific questions must be asked, of 
the employer by correspondence before 
the position is sufficiently defined. Of 
course, practically all inquiries specify 
the subjects and grade levels to be 
taught. The next most common specifica- 
tion for positions on the secondary level 
relates to the sex of the candidate. From 
his experience with the employer and 
knowledge of the community the execu- 
tive fills in the questions left by the 
inquiry in an attempt to get as complete 
a picture as possible before making 
nominations. 

As the executive searches his list of 
available candidates, he has to keep in 
mind several considerations. Specifica- 
tions mentioned by the employer in the 
inquiry or inferred by the executive 
from the description of the position or 


the school are important whether they . 


appear to the executive as reasonable or 
unreasonable demands. In _ addition, 
there may be considerations, unacknowl- 
edged by the employer, which are known 
to the executive because of experience 
with the employer, or which are assumed 
because of knowledge of the community. 
Finally, the rewards offered by the posi- 
tion must be considered. The salary, the 
opportunity for advancement, the oppor- 


tunity to teach certain specialties, the 
opportunity to work under favorable 
conditions of supervision and adminis- 
tration, and other kindred factors are of 
utmost importance in the mind of the 
candidate and therefore are carefully 
considered by the executive. If informa- 
tion about these factors is lacking, it 
must be searched for and estimates made 
of the salary to be paid. 

It is these rewards that chiefly deter- 
mine the quality of the candidates that 
are nominated. Superior candidates are 
sometimes nominated, with the permis- 
sion of the candidate, for inferior posi- 
tions, but not generally, and it is seldom 
that inferior candidates are nominated 
for superior positions. The influence of 
the rewards offered by the position as 
a factor in the selection process is de- 
cidedly in the direction of a reduced 
range of general quality among the 
nominees. The tendency is to concen- 
trate the candidates within a relatively 
reduced range of general calibre. Never- 
theless,. there are counteracting forces 
that keep the range from becoming 
unusually narrow. For his judgment 
with respect to the candidate’s general 
quality the executive has all three 
sources of information: impressions from 
interviews, facts from the biographical 
forms, and evaluations to be found in 
the recommendations. 

Ordinarily the candidate to be nomi- 
nated must meet the specified require- 
ments, and it is, desirable that he meet 
the specified preferences also; not so 
much because of the importance of such 
requirements and preferences, even in the 
minds of employers, as because of the fact 
that frequently there are plenty of can- 
didates who do meet the specifications 
exactly and as long as that is true, there 
is no reason for nominating candidates 
who do not. if, however, the specifica- 























tions are large in number or unusual in 
character and the number of available 
candidates has been reduced by other 
considerations, an executive may nomi- 
nate candidates who do not meet the 
specifications at every point, especially if 
ihe rewards offered by the position are 
limited. Sometimes the executive will 
nominate a candidate who fails to meet 
the specifications at one point if he is 
especially strong at other points, with 
the hope that the employer will consider 
the candidate in spite of his failure to 
meet all of the specifications. Sometimes 
andidates may even be nominated who 
‘ail to meet the subject matter require- 
ments if the position is to teach a variety 
of subjects. This happens because the 

xecutive knows from experience that 
‘he employer may not be able to find 
andidates who have just the combina- 
ion of subject matter backgrounds de- 
sired, and in such an event, the candi- 
lates nominated may be considered. In 
spite of exceptions, however, in general, 
nly candidates who meet the specifica- 
ions of the inquiry are nominated. Speci- 
fications may relate to many qualifica- 
tions. ‘The more usual items are these: 
field of primary specialization, fields of 
econdary specialization, sex, number of 
ears of teaching experience, ability in 
extra-curricular activities. The presence 
of specifications for the position in all 
‘ikelihood reduces the number of candi- 
‘ates available for nomination, but it is 
o be doubted if, in general, the range 
{ general quality among the candidates 
; thereby reduced. 

Unreasonable requirements are also 
usually unspecified, .but not all unspeci- 
fied requirements are unreasonable. Al- 
hough it is not always specified, it may 


assumed, for instance, that most em- 


loyers will prefer a candidate with 
ching experience, other factors being 
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equal. Unspecified requirements may or 
may not be acknowledged by the em- 
ployer. Many of the latter and some of 
the former will be the result of individual 
predilections that the employer either has 
as a person or feels forced to adopt be- 
cause of the viewpoints of his commu- 
nity. It is important to note that the op- 
eration of such factors can rarely be 
demonstrated; however, the fact that such 
factors operate is accepted generally by 
placement workers. Unspecified require-: 
ments, whether acknowledged by the 
employer or not, and whether reasonable 
or not, are like specifications in that while 
they undoubtedly reduce the number of 
candidates available for nomination, they 
probably do not reduce the range of gen- 
eral quality among the candidates nomi- 
nated. This would seem to be true even 
for prejudices. 

Requirements of the type discussed so 
far, whether specified or not, whether ac- 
knowledged or not, and whether reason- 
able or not, are alike in that they are 
particular. The requirement may be that 
the candidate have had a period of study 
in France; it is particular. The prefer- 
ence may be for a married woman; it 
is particular. The degree to which the 
candidate meets this type of requirement 
is evaluated both by the placement ex- 
ecutive and the employer with factual in- 
formation; and that is secured from the 
items of information (non-ratable) in the 
recommendations, from interviews, and 
from the biographical forms, but prin- 
cipally from the last. Therefore, in the 
next chapter, the effects of this informa- 
tion on the candidate’s employment 
chances are referred to as extra-recom- 
mendation factors. 

The other type of requirement of the 
employer is general. The two principal 
general requirements are that the candi- 
date make a favorable impression in an 
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interview and that the evaluations of the 
candidate made by others in the recom- 
mendations be favorable. 

Undoubtedly it would have been pref- 
erable if the varying impressions made by 
the candidates in the interviews held 
could have been taken into account, but 
that was found to be impossible. The 
Office of Placement Service records were 
marked only when it was known that an 
interview had been held within the Of- 
fice itself, no attempt being made to sys- 
tematically record whether interviews 
had taken place outside of the Office. 
Evidence that an interview was sched- 
uled could not be taken as evidence that 
it had been held. For example, the cor- 
respondence revealed situations in which 
the executive had arranged for inter- 
views, but for various reasons they had 
not been held. 

Sometimes there was evidence that the 
successful candidate was interviewed but 
that none of the others were. This usually 
means that the credentials had been stud- 
ied by the employer and the best candi- 
date chosen for an interview with the 
thought that if the interview showed the 
candidate to be satisfactory, he would 
be offered the position. In such situations 
the interview is not a factor to be taken 
into account in the comparison of suc- 
cessful with unsuccessful candidates. In- 
stead it must be considered a basis for 





checking a selection which had been pre- 
viously made on the basis of other fac- 
tors. Eleven such known cases in the data 
were not considered enough. to warrant 
further study of this group by itself. 

For every position a list of candidates 
is nominated by the placement executive. 
In this study the number varies from 2 
to 8. On the basis of information ob- 
tained from interviews with the candi- 
date, from the biographical forms, and 
from the recommendations, the candi- 
dates, in the opinion of the placement 
executive, are fairly matched with respect 
to the calibre of candidate appropriate 
to the salary and other rewards offered 
by the position, are alike in that they all 
meet or reasonably approximate the 
specifications of the inquiry, and are all 
serious competitors in that unspecified 
requirements likely to influence the em- 
ployer and known to the executive have 
been taken into account. One of the list 
is successful. If it is assumed that candi- 
dates unable to meet unspecified require- 
ments had been eliminated or that such 
factors had not operated; that the candi- 
dates were evenly matched with respect 
to the specifications for the position; and 
that the candidates made equally good 
impressions in any interviews held, then 
the successful candidate should be found 
to have the best recommendations. 














CHAPTER 5 


ExTRA-RECOMMENDATION FACTORS AND PLACEMENT SUCCESS 


ECOMMENDATION excellence is only 
R one of several factors determining 
placement success. Actually a relation be- 
tween placement success and recommen- 
‘ation quality as one of the constituents 
‘! recommendation excellence might ex- 
ist but be obscured by extra-recommen- 
‘ation factors operating in such a way 
as to conceal the relation. It would be 
better if the relation could be studied 
under conditions in which the effects of 
other factors exclusive of the recommen- 
dations were equivalent, i.e., when all of 
ihe candidates had the same or equiva- 
‘lent status with respect to all of the re- 
quirements of the employer. Such ideal 
situations are impossible to obtain, of 
ourse, so the practical course was taken 
§ working with job-situations in which 
it was not known that other factors were 
‘ecisively unequal. If nominees whose 
candidacies might have been affected un- 
equally (for that job-situation) by extra- 
recommendation factors were eliminated, 
in the remaining job-situations, influ- 
nces preventing recommendation evalu- 
tions of superior quality from securing 
positions for the candidates for whom 
ihey were written would be reduced to a 
minimum. 

Two comparisons of the recommenda- 
tions of the successful candidates. with 
those of their unsuccessful competitors 
were made, the first, with all of the job- 
situations and all of the candidates, and, 
the second, with only those job-situations 
and candidates that remained after the 
elimination of nominees whose candi- 
dacies appeared to be affected unequally 
by extra-recommendation factors. In or- 
der to determine which candidates to 


Y 





eliminate, an individual study was made 
in case history fashion of all of the avail- 
able data for each job-situation. Not only 
were the facts on record reviewed, but 
the executive who handled the inquiry 
and any other executives familiar with 
the employer or the coramunity were 
consulted for information as to the pos- 
sible operation of factors involved. The 
decision to retain or eliminate a candi- 
date was arrived at only on the basis of 
the total picture and after the whole job- 
situation had been outlined as in the 
studies in the Appendix. 

The fact that it is the status of the 
candidate with respect to the require- 
ment that constitutes the factor of inter- 
ference will bear emphasis. This means 
that in every job-situation as far as pos- 
sible the requirements for the position, 
both reasonable and unreasonable and 
both acknowledged and unacknowledged, 
must be ascertained and the status of each 
candidate with respect to each require- 
ment must be examined. If the status of 
a candidate with respect to a require- 
ment operates as a factor which by itself 
(i.e., apart from the recommendations), 
or in combination with other such fac- 
tors, could reasonably be judged to have 
caused the success or failure of the can- 
didate, the true relation between recom- 
mendation quality and placement suc- 
cess is obscured or interfered with, In 
other words, an interfering factor is pres- 
ent whenever a status operates to the un- 
equal advantage of a successful candi- 
date or to the unequal disadvantage of 
an unsuccessful candidate. Decisive in- 
terference in a given job-situation can 
never be proved, of course, but, when- 
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ever there was a reasonable possibility 
of decisive interference, the candidate 
was eliminated. If the successful candi- 
date or so many unsuccessful candidates 
were eliminated that not a single com- 
parison between successful and unsuccess- 
ful candidates remained, the job-situation 
was automatically eliminated. 

Not all facts about a candidate out- 
side of the evaluations of the recom- 
mendations are possible sources of inter- 
ference, however. Some of such outside 
facts are represented in the evaluations 
and therefore have the opportunity to 
influence the relation between recom- 
mendations and placement success in that 
way. The effect or influence of the fact 
is felt through the medium of the rec- 
ommendations because the fact is con- 
comitant with rather than independent 
of the recommendation evaluations. 

Membership in honorary societies may 
be taken as an example. If a candidate is 
a member of an honorary society, pre- 
sumably it is because of his accomplish- 
ments as a student in education. Cer- 
tainly these accomplishments will be re- 
flected in the evaluations found in the 


recommendations. This relationship is 


one of correlation. If then in a certain 
job-situation the successful candidate is 
a member of an honorary society and 
none of the unsuccessful candidates are, 
it is not necessary to assume that his 
membership constitutes an interfering 


factor. If that were assumed, the candi-. 


date would be eliminated and a perfectly 
legitimate comparison in which the more 
able nominee might be expected to be 
the successful candidate would be elimi- 
nated. A relation cannot be found by 
eliminating, for no reason, those com- 
parisons in which it is most likely to 
occur. 

A record of one year of experience in a 
position provides another example. Not 


always, but frequently such records indi- 
cate failure in the position. Inasmuch as 
a record of failure in a position would 
influence an employer, it might seem that 
such a record would be an interfering 
factor. However, if there has been failure, 
it should be represented in the evalua- 
tions in the recommendations, and 
through that medium will adversely af- 
fect the candidate’s employment chances. 
One year of experience is a factor, but 
a correlative rather than an interfering 
factor. 

Whenever the status of both the suc- 
cessful and the unsuccessful candidate 
was the same with respect to a require- 
ment, it was judged that no interference 
was involved. This was true regardless 
of whether from the nature of the re- 
quirement the status was judged to be 
of advantage or disadvantage. In Job- 
Situation No. 22* both Candidates A 
and C disavowed any church connection. 
Such a status would ordinarily be con- 
sidered disadvantageous, and if Candi- 
date A, who was successful, had acknowl- 
edged affiliation with a church, Candi- 
date C would have been eliminated. But 
with both candidates having the same 
status, no interference could be imputed. 

In general, whenever two opposing 
candidates have a different status with 
respect to the same requirement there is a 
possible source of interference. Whether 
interference is judged to be present de- 
pends upon the particular requirement, 
the particular status, and whether the 
candidate was successful or unsuccessful. 
If the status is presumably disadvan- 
tageous but the candidate was neverthe- 
less successful, no interference was con- 
sidered to be involved. In Job-Situation 
No. 37, for example, Candidate A was 
successful in spite of the fact that he did 


‘Outlines of job-situations referred to in this 
chapter are presented in the Appendix. 
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not meet the specified preference for a 
candidate with ability to direct athletics, 
whereas the other two candidates did. 
Although his inability to direct athletics 
must have been to his disadvantage, 
other factors, among which must be in- 
cluded the quality of his recommenda- 
tions, were sufficiently strong to over- 
come that disadvantage in the judgment 
of the employer. 

If the status is presumably advanta- 
eeous but the candidate was not success- 
ful, no interference was considered to be 
involved. In Job-Situation No. 12 one 
candidate had had two years of teaching 
experience whereas his competitor had 
had none. Nevertheless no interference 
was involved on that point because the 
experienced candidate (B) was not suc- 
cessful. 

Whenever two opposing candidates 
have a different status with respect to the 
same requirement, interference may be 
involved either because a successful can- 
didate may have been aided by an ad- 
vantageous status or because an unsuc- 
cessful candidate may have been injured 
by a disadvantageous status. ‘The success 
of Candidate A in Job-Situation No. 24 
which may have rested on the fact that 
he was a “local candidate” illustrates the 
first case. For this study there is no way 
of determining whether the candidate 
was successful because of local influence, 
because of the superior quality of his 
recommendations, or for other reasons; 
but when the possibility that he was suc- 
cessful because of local influence is as real 
and substantial as it is in that job- 
situation, the fact that he was a “local 
candidate” must be considered an inter- 
fering factor, The second case is illus- 
trated by the elimination of Candidate 
B in Job-Situation No. 22 because the 
adequacy of her subject matter prepara- 
tion was doubtful. Apart from the qual- 


ity of her recommendations, this in- 
adequacy, of itself, might have been a 
decisive factor in her failure to secure the 
position. 

Whenever it was impossible to judge 
with confidence from the nature of the 
requirement whether a given status was 
advantageous or disadvantageous, the sta- 
tus of the successful candidate was pre- 
sumed to be one of advantage. The use 
of this rule was frequently required when 
candidates of different church affiliations 
were competing with each other. ‘Thus in 
Job-Situation No. 37, inasmuch as A and 
B, opposing candidates, are of different 
religious faiths, and inasmuch as the 
predilections of the employer and the 
community were unknown, there is a 
possibility that Candidate A received the 
position because of his religion, that 
Candidate B lost the position because of 
his religion, or that both influences oper- 
ated—in any one of these cases interfer- 
ence is involved. 

However, the fact that the successful 
candidate has a different status from that 
of the unsuccessful candidate does not 
necessarily mean that the status of the 
unsuccessful candidate is one of dis- 
advantage. It depends upon the status 
of each and the requirement. For in- 
stance, in Job-Situation No. 18 the status 
of Candidate B, a Protestant, was not 
presumed to be disadvantageous just be- 
cause it is different from that of Candi- 
date A, who stated that she was not a 
member of any church. Rather it was 
presumed that Candidate A was success- 
ful in spite of her status with respect 
to church affiliation and that Candidate 
B’s status was not a reason for her failure. 

Failure to meet both reasonable and 
unreasonable demands on the part of an 
unsuccessful candidate must be consid- 
ered as possible sources of interference. 
In the same way success in meeting both 
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reasonable demands as well as prejudices 
on the part of the successful candidate 
must be considered sources of interfer- 
ence. In either case a factor operates in- 
dependently of the influence of the evalu- 
ations of the recommendations, and thus 
obscures the relation between recommen- 
dations as judged by the quality of the 
evaluations therein and placement suc- 
cess. 

The following requirements were spe- 
cifically kept in mind by the author 
while studying the factors operating in 
each individual job-situation. The effect 
of different possible status under each 
will be clear in some cases without ex- 
planation. Whenever it seemed neces- 
sary, the assumptions made regarding 
the effect of different possible status are 
indicated. 


1. The specifications the employer men- 
tioned in the inquiry. 

In every case possible, the specifications 
listed on the inquiry record form were 
checked with the letter in which the in- 
quiry was received. In cases in which the 
inquiry was received by telephone or 
through a personal interview, a complete 
check could not be obtained. 

One of the demands most likely to be 
specified is ability in a particular extra- 
curricular activity. Ability in  extra- 
curricular activities has not been included 
in the list below because, if it is not speci- 
fied, it may be presumed to be un- 
important, whereas the other factors 
listed are important whether specified or 
not. 

2. The sex of the candidate. 

It was assumed that there was no pref- 
erence for either sex unless a require- 
ment was mentioned in the inquiry or 
could reasonably be inferred from the 
type of school. 

3. The age of the candidate. 

Unless there was evidence to the con- 
trary, it was assumed that age might be a 
factor against candidates 40 years of age 
or over who were in competition with 
younger candidates. 


4. The age of the candidate in relation to 
the number of years of experience. 

It was assumed that a candidate over 
go years of age who had changed his 
vocation and had had very little teaching 
experience might have: been at a dis- 
advantage. 

5. Religion stated by the candidate. 

In general, unless an opinion of an 
executive who knew the employer and/or 
the community was evidence to the con- 
trary, the possibility that religion was a 
factor was assumed when the religion of 
the successful candidate was in one group 
and the religion of the unsuccessful can- 
didate was in a different group. The fol- 
lowing scheme of grouping was used: 

a. Protestant, Unitarian, Christian Scien- 
tist, Non-Sectarian, Christian, No Pref- 
erence. 

- Roman Catholic. 

Hebrew. 

. Other. 

The word none used by the candidate, 

or the space left blank. 

6. The number of years of teaching experi- 
ence. 

The year 1936-37 was included in the 
number of years of teaching experience 
if the candidate was then employed and 
the work of the position was to start 
after the school year had closed. In every 
case the same basis of calculation was 
used for all of the candidates for a given 
position. 

Practice teaching of any variety done 
while the candidate was a student was 
not counted as teaching experience. 
Teaching as a substitute was given sepa- 
rate mention, as was teaching in summer 
sessions. Experience as a New York City 
teacher-in-training was considered as reg- 
ular teaching experience. Teaching ex- 
perience on any level of the educational 
system, in either public or private schools, 
in the United States or abroad, was 
counted. Special types of teaching ex- 
perience were noted and are mentioned 
in the outlines to be found in the Ap- 
pendix. 

Teaching experience was considered a 
factor of advantage and lack of experi- 
ence a factor of disadvantage regardless 
of whether experience was mentioned as 
a specification. 
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. If the position was in a private school, 


the number of years of teaching ex- 
perience in private schools. 

Private school teaching involves ad- 
justments not always demanded in public 
school teaching and successful experience 
in private schools is evidence of ability 
on the part of the candidate to make 
those adjustments. 


. Unusual features of the teaching ex- 


perience of the candidate in relation to 
the demands of the position. 


. The field of primary specialization in 


relation to the demands of the position. 
The fields of secondary specialization in 
relation to the demands of the position. 


. In job-situations where it might have 


been a factor, marital status. 

In the case of public schools, it was 
assumed that there was no bias either 
for or against married men, single men, 
or single women. In the case of private 
schools with large enrollments of board- 
ing students, it was recognized that be- 
cause staff members are frequently as- 
signed dormitory duties, there is often a 
preference for unmarried candidates. It 
was assumed that if there was a prefer- 
ence for married women, the inquiry 
would so state; otherwise the possibility 
of bias against married women was con- 
sidered. 


. In job-situations where it might have 


been a factor, the number of children. 

In the case of married women candi- 
dates, and men candidates for positions 
in private schools, this may have been a 
factor. 


g. In job-situations where it might have 


been a factor, the degree held. 


. In the case of positions in private schools, 


whether the compensation included 
maintenance. 

This is important because married 
candidates are frequently not in a posi- 
tion to live in a dormitory. 

Foreign nationality or background. 

This is frequently an advantage if the 


16. 


17. 
18. 


19. 


20. 
21. 


22. 


23. 


24. 


position is to teach the native language; 
otherwise it may be a disadvantage. 
Record of divorce. 

Physical handicap. 

Expressed lack of interest in the age 
group of the school or unit in which the 
position is open, or expressed interest 
in the age group of another level. 
Expressed extreme political or social 
views. 

State certification requirements. 

Special recommendation by the Office of 
Placement Service. 

Whether the candidate was a “local can- 
didate.” 

A “local candidate” is one who either 
has local influence in his favor or against 
whom there might be prejudice because 
he came from. the locality and the em- 
ployer had a practice of not employing 
local candidates. 

Whether the credentials were sent upon 
request of the employer. 

Such a request does not represent evi- 
dence that the employer has special in- 
terest in the candidate, but it does 
constitute evidence that there is no spe- 
cial disinterest. 
Whether there is a record of an interview 
at the request of the employer. 

Such a record constitutes positive evi- 
dence that the employer was interested 
enough in the candidate to interview 
him. 


If all of the candidates were known 


to have been interviewed, that fact was 
noted and indicated in the outline made 
for each job-situation, although the 
job-situation was not eliminated on that 
account. As has already been explained, 
in this study, interviews represent an un- 
controlled factor. 


Sargent’s Handbook (15) was taken as 


an authority for information regarding 
private schools. 





CHAPTER 6 


THE COMPARISON OF SUCCESSFUL WITH UNSUCCESSFUL CANDIDATES 


F A TOTAL POPULATION of scores ho- 

mogeneous in character is divided 
into random groups, the means for those 
groups would be expected to differ from 
each other, but within relatively con- 
fined limits. If, however, the total popu- 
lation is really a mixture of scores from 
two or more populations with different 
means, and if the groups are so chosen 
that scores from one population pre- 
dominate in one group and scores from 
another population predominate in an- 
other group, and so on, the means of the 
groups should differ relatively widely 
from each other. Examination of the 
variation among the means of a set of 
groups, then, ought to indicate whether 
the groups can be considered to be ran- 
dom samples from one homogeneous 
population. In general, analysis of vari- 
ance, which is the method chiefly used 
in this study, is a method for evaluating 
the null hypothesis which states that each 
group is a random sample from the same 
homogeneous population. If the groups 
cannot be shown to be random samples 
from the same population homogeneous 
with respect to the characteristic being 
studied, they have not been demon- 
strated to be the same with respect to 
that characteristic. The analysis is ac- 
complished by a statistical procedure 
which compares the variation among 
the means of two or more groups with an 
estimate of that variation based upon the 
variation of the scores within the groups. 
When the variation among the means is 
relatively large in relation to this esti- 
mate, the groups cannot be considered to 
be random samples from the same ho- 
mogeneous population; when the vari- 


ation is relatively small, the groups may 
be so considered. 

If the groups are, in fact, from the 
same homogeneous population, any ob- 
served differences between the groups 
are accidental and to be accounted for by 


- the fact that samples rather than whole 


populations have been dealt with. If the 
groups are, in fact, from different popu- 
lations, the observed differences are sig- 
nificant inasmuch as they reflect the pres- 
ence of real or true differences between 
populations. If the groups can reason- 
ably be considered to be from the same 
population, it is safest to assume that 
they are, because the object of the testing 
is to discover whether there are differ- 
ences between the groups and very prop- 
erly the burden of proof rests upon the 
demonstration that the groups are de- 
rived from different populations. If the 
groups cannot reasonably be considered 
to be from the same population, they 
may be presumed to be from different 
populations, in which case the differ- 
ences between the groups are significant, 
i.e., statistically significant, or reliable 
differences, or real or true differences. 
The test, then, is: Can the groups rea- 
sonably be considered to be from the 
same homogeneous population? If they 
cannot, a significant or reliable difference 
may be accepted. 

In either case it is a matter of whether 
groups “can reasonably be considered to 
be” of a certain character. This is the 
point at which inference and judgment 
enter the process. Starting with certain 
assumptions about the general character 
of the data, the probability that the ob- 
served differences, which are differences 
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of a certain size and direction, would 
occur as a result of chance errors if the 
null hypothesis were true can be calcu- 
lated with rigorous mathematical accu- 
racy. That probability, under the as- 
sumptions of the method—all reasonable 
assumptions—is a fact. But the inference 
from the fact is a matter of judgment. If 
the probability of occurrence (under the 
null hypothesis) of differences which did 
actually occur is very small, almost any- 
one would discard the hypothesis. ‘The 
only alternative would be to assume that 
a very unlikely event had, indeed, oc- 
curred. On the other hand, if the prob- 
ability of occurrence of the differences 
which did actually occur is large, almost 
anyone would accept the hypothesis. 
Whenever the null hypothesis is rejected, 
even though the probability of occur- 
rence of the observed differences is small, 
there is some danger that the unlikely 
event has, in fact, occurred and that 
the hypothesis does actually describe the 
population and should therefore be ac- 
cepted. Whenever the null hypothesis is 
accepted, even though the probability of 
occurrence of the observéd differences 
is large, there is a parallel danger that 
the null hypothesis has been wrongly ac- 
cepted when some contrary hypothesis 
is, in fact, true. Thus one may actually 
have at hand significant differences and 
fail to recognize the fact. 

From this it can be seen that the 
selection of a probability to mark the 
dividing point between acceptance and 
rejection of the hypothesis is a problem. 
If the probability selected is extremely 
small, the danger of failing to declare 
as significant differences that actually 
are, increases. Consequently an unduly 
rigorous criterion must be avoided. If 
the probability selected is relatively 
large, the danger of declaring as signifi- 
cant differences which are not, increases. 


) 


| 


Inasmuch as for research purposes it 
is usually better to fail to find signifi- 
cant differences than to declare as sig- 
nificant differences which are not, a 
probability whith is relatively small is 
usually chosen. The 5 per cent point and 
the 1 per cent point are commonly used 
and are referred to as levels of confi- 
dence. At the 5 per cent level, the prob- 
ability of occurrence of differences of the 
size in question or larger is 5 times in 
100 under the null hypothesis; at the 
1 per cent level, once in 100 times. If 
the probability of occurrence of differ- 
ences of a given size is only once in 100 
times under the null hypothesis, and dif- 
ferences of that size are observed, prac- 
tically all research workers would reject 
the hypothesis. Such an event is too un- 
likely, and one is obliged to choose an- 
other hypothesis under which the event 
is more likely. If the probability of oc- 
currence of differences of a given size is 
greater than 5 times in 100 under the null 
hypothesis, and differences of that size 
are observed, ordinarily conservative re- 
search workers will accept the hypothe- 
sis because although the event is unlikely 
it is better to be safe and reject differ- 
ences as not significant than to accept as 
significant differences which are not and, 
perhaps, have to regret the action later. 
There is nothing sacred about either 
point, however. 

The result of analysis of variance pro- 
cedures is a ratio called F. For any given 
number of groups of specified sizes, there 
is a value of F for any given level of 
confidence under the null hypothesis. 
Values of F at several levels are available 
in printed tables. Interpretation of an 
obtained F involves comparing it with 
the table value of F for groups of the 
given number and sizes at the selected 
level of confidence. The larger the ob- 
tained value of F compared with the 
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table value of F, the more extreme or 
smaller is the percentage signifying the 
level at which the differences are just sig- 
nificant and the smaller is the probabil- 
ity that the obtained differences would 
occur as a result of chance errors when 
the null hypothesis is true, and vice 
versa. When the obtained F is larger than 
the table value of F, the differences may 
be accepted as significant with even 
greater confidence than the table level 
indicates. In this report the obtained 
value of F and the table values of F 
(17, pp. 184 ff.) for both the 5 per cent 
and the 1 per cent level of confidence 
will be given. With this information, in 
each case, the reader may form his own 
opinion as to whether the hypothesis 
should be accepted or rejected. 

The data explored consist of 1035 
recommendations written for 183 candi- 
dates who were nominated for 58 differ- 
ent positions in 8 different subject mat- 
ter categories. Of the 183 candidates, 58 
were successful and 125 are their un- 
successful competitors. The number of 
candidates per vacancy varies from 2 to 8. 
Each recommendation has a score repre- 
senting its quality derived by the pro- 
cedure explained in Chapter 3. 

These recommendation scores vary 
from recommendation to recommenda- 
tion and a measure of such variation can 
be found for any given group of recom- 
mendation scores. For example, if the 


recommendations of a single candidate . 


are taken, the variation in those scores 
can be ascertained. At the same time if 
those scores are averaged, a mean score 
for the candidate is obtained. And these 
candidate scores vary from one to an- 
other giving a second measure of vari- 
ation. From this and similar sets of rela- 
tionships, two measures of variation may 
be obtained, a measure of variation 
within the groups of scores from which 


the group means have been derived, and 
a measure of variation among the group 
means that have been derived. Analysis 
of variance methods rests upon a com- 
parison of these two measures of vari- 
ation, and for purposes of elementary ex- 
position, F may be thought of as the 
ratio between them. 

A similar pattern results if the rec- 
ommendation scores are grouped on 
other bases. If the recommendation scores 
of all of the candidates nominated for a 
single position are grouped together and 
averaged, a mean is obtained which is a 
position score. Again there is variation 
in the recommendation scores that make 
up the position mean or score and vari- 
ation in the position means or scores 
themselves. If the recommendation scores 
of all candidates majoring in a given sub- 
ject are grouped together and averaged, 
a subject score is obtained. Likewise 
there will be variation among these sub- 
ject scores and among the recommenda- 
tion scores of which each is made up. 
Finally if the recommendation scores of 
all candidates are grouped according to 
whether the candidate was successful or 
unsuccessful and averaged for each 
group, means will be obtained which 
may be taken as scores for the respective 
groups. These two means will differ and 
a measure of the variation or difference 
between them may be calculated. And 
again there will be variation among the 
recommendation scores of which the 
group mean is made up. The basic unit 
throughout has been the score of a single 
recommendation. Similar groupings 
could be obtained with the candidate 
mean as a score and basic unit. In each 
case the variation among the means is 
to be compared with the variation within 
the groups of scores from which the 
means are obtained. Roughly speaking, 
if the variation among the means is 
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small, the groups whose means have 
been evaluated may be presumed to be 
from the same population; if the vari- 
ation among the means is large, the 
groups must be presumed to be from 
different populations, which is equiva- 
lent to saying that the differences be- 
tween the groups are significant. More 
precisely speaking, the evaluation of the 
variation among the means by compari- 
son with the variation among the scores 
centers in the F ratio which is not very 
different from 1 when the groups are 
random samples from the same popu- 
‘ation, and tends to deviate from 1 in 
the direction of greater magnituuce when 
the groups are from different popula- 
tions. 

The procedure with which the recom- 
mendations were analyzed and the data 
recorded is explained in Chapter 3. In 
this process a quality rating was esti- 
mated for each evaluation of the candi- 
date by the writer of the recommenda- 
tion. ‘The mean of these quality ratings 
was calculated for each recommendation 
and assigned as a score representing rec- 
ommendation quality. In the case of a 
series Of evaluations of the same trait, 
the rating was based upon the impres- 
sion made by the impact of the whole 
series; and in the calculation of the mean 
for the recommendation, such a rating 
was weighted by the number of evalu- 
ations in the series. All of the analyses of 
the association between placement suc- 
cess and recommendation characteristics 
here reported were made with recom- 
mendation quality as the measure of 
recommendation excellence.® 

* Plans were made for investigating the asso- 
ciation between placement success and length 
of recommendation, but because preliminary 
exploration revealed little promise, the work was 
discontinued. With a group of candidates some 
of whom were later dropped before the analyses 


were made with regard to recommendation 
quality, the differences among candidate scores 


The first analysis was made grouping 
the recommendations according to the 
candidate for whom they were written, 
yielding 183 groups of recommendations, 
each group being all of the recommenda- 
tions of a given candidate. There is a 
relatively wide variation among the 183 
candidate scores, as can be seen from 
Table 1, and the obtained F is significant 


TABLE 1 


Relation of the variation among candidate scores 
to an estimate of this variation based upon the 
variation among recommendation scores 





Sum of Mean 


Squares Square 


Variation df. 





Among candidate 

scores 182 
Among recommenda- 

tion scores grouped 


76.478" .42021* 





by candidates 852 208.703 .244957 
Total 1034 285.181 
.42021 F (05) =1.19 
F = 7344957 71-7154 or 1.72 F (or) =1.28 





* The sum of the quality ratings of the indi- 
vidual evaluations for a recommendation, an 
approximate number, usually had two digits but 
never more than two, and inasmuch as the num- 
ber of evaluations was exact, the mean (recom- 
mendation score), an approximate number, 
usually had two significant digits and for uni- 
formity was always written with two digits. 
Usually one decimal place in the mean was sig- 
nificant. However, to prevent unduly large round- 
ing errors and for uniformity, in every instance, 
two additional decimal places were carried 
throughout the computation of the Sum of 
Squares and the Mean Square. The number of 
significant digits in each is two less than the 
number recorded. The number of significant 
digits in the obtained F is the number finally 
recorded. 





were not significant at the 5 per cent level. This 
indicates that length of recommendation, as 
measured by the number of Inventory items 
commented upon not unfavorably, did not 
even distinguish reliably among the candidates 
themselves. 

Plans were also made for investigating the 
relation between individual traits and _place- 
ment success, but because an exhaustive study 
would in reality have amounted to a second 
investigation, these plans were postponed. The 
chi-square test was applied to determine whether 
in likely looking cases the frequency of mention 
of individual Inventory items was significantly 
greater for successful candidates than for un- 
successful candidates. A discrepancy not to be 
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at a point well beyond the 1 per cent 
level. Evidently the null hypothesis, that 
all of the candidate scores are derived 
from random samples from the same ho- 
mogeneous population of recommenda- 
tions, is untenable, and the differences 
among the candidate scores must be con- 
sidered to be significant. This means 
that the candidate scores differ reliably 
from each other, and that the recom- 
mendations discriminate among the can- 
didates. This result is what might be 
expected, but the fact that it was ob- 
tained validates the recommendation 
scores as estimates of recommendation 
quality. Failure of recommendation 
scores to discriminate reliably among the 
candidates would have indicated either 
that the candidates did not differ reli- 
ably from one another, or that the pro- 
cedure for analyzing the recommenda- 
tions was not sufficiently diagnostic, or 
both. While there is reason to believe 
that the procedure for analyzing the rec- 
ommendations was not maximally diag- 
nostic, the finding that the recommenda- 
tion scores obtained from it discriminate 
among the candidates validates the pro- 
cedure as a useful instrument. 

The principal relationship this study 
undertakes to explore is that between 
recommendation quality and placement 
success. The first work on the question 
that the author could find reported was 
done by Morrisett who investigated this 
question, “How and to what ‘extent do 
letters of recommendation written about 





reasonably accounted for by chance errors was 
found for only one of the Inventory items, but 
with the total number of frequencies being 42 
and with theoretical frequencies as small as 4, 
even the appropriateness of the test is in some 
doubt. More exact tests were not applied be- 
cause the crude application of the chi-square 
test gave so little promise of significant results. 

*In this connection it is worthwhile to note 
that the number of Inventory items commented 
upon not unfavorably did not distinguish reli- 
ably among the candidates. 


candidates selected for high school teach- 
ing positions vary from those written 
about candidates who were not selected 
for similar positions in the same school 
system?” (13, p. 7). In his research Mor- 
risett analyzed 1851 letters of recommen- 
dation with the technique and list of 
traits used by Charters and Waples in 
the Commonwealth Study referred to 
earlier (see pp. 13-14). In addition he 
analyzed 195 letters of recommendation 
written for candidates who were unsuc- 
cessful in securing positions in the same 
school systems in which 40 of the total 
number of candidates for whom the 1851 
letters were written did secure positions. 
His conclusions are quoted here. “The 
results of this investigation indicate that 
letters of recommendation written about 
candidates who were selected for high 
school teaching positions do not vary in 
any observable degree from letters writ- 
ten about candidates who were not se- 
lected for similar positions in the same 
school systems. ‘The same items of infor- 
mation appeared with relatively the 
same frequency in the letters written 
about the teachers who were not em- 
ployed as in the letters about those who 
were employed” (13, p. 118). 

At the Sixth Annual Fall Conference 
of the National Institutional Teacher 
Placement Association Evan R. Collins 
(9) reported on a study made at Harvard 
University in which written recommen- 


dations of “better candidates’’ were com- 


pared with those of “‘less desirable candi- 
dates.”” From the brief report available 
it is not clear whether the “better can- 
didates” were successful and the “less de- 
sirable candidates” unsuccessful, or not. 

As the employer’ probably thinks in 
terms of candidates rather than individ- 
ual recommendations, the candidate 
score is probably the more appropriate 
unit to employ in the comparison of 
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successful candidates with their unsuc- 
cessful competitors. Since the candidate 
score has been shown to distinguish re- 
liably among candidates, it is a depend- 
able unit. Had that not been the case, no 
further analysis would have been justi- 
fied. 

‘The mean candidate score for the suc- 
cessful group is 2.451 and the mean for 
the unsuccessful group is 2.518, indicat- 
ing that the successful candidates have 
slightly better scores, The difference be- 
tween these two means is slight, .067 
being less than one-tenth of an interval 
on the Quality Rating Scale. Neverthe- 
less, if the variation between the two 
means is great relative to the variation 
among the scores of which the means 
are composed, the difference between 
the two means is significant, small as it 
is. Reference to Table 2 will show that 


TABLE 2 


Relation of the variation between the mean 
candidate score for successful and for unsuccess- 
ful candidates to an estimate of this variation 
based upon the variation among 
candidate scores 











Sum of Mean 
Variation d.f. Squares Square 
Between — success-cate- 
gory means I 179 .179 
Among candidate scores 
grouped by success 
categories ° 181 14.403 .079575 
Total 182 14.582 
i -179 F (05) =3.89 
F = o70575 = 2:25 or 2 F (01) =6.76 





such is not the case. The obtained F 
cannot be considered significant at the 
5 per cent level. At first glance this 
might seem to end the matter, but there 
are other possibilities to be considered, 

The magnitude of F is related to two 
variations, one, the variation in the 
group means represented by the numer- 
ator of the fraction, and, two, the vari- 
ation in the scores within groups repre- 


sented by the denominator. If either of 
these parts of the fraction contains ele- 
ments that are not in the variation in- 
tended to be represented, that part is 
spuriously enlarged and as a result the 
F is either spuriously inflated or de- 
flated. 

If the 183 candidates are grouped ac- 
cording to the position for which the 
candidate was nominated, 58 groups with 
58 means or position scores are produced 
which are significantly different at the 
5 per cent level as is shown in Table 3. 


TABLE 3 


Relation of the variation among position means 
derived from candidate scores to an estimate of 
this variation based upon the variation 
among candidate scores 











Sum of Mean 

Variation d.f. Squares Square 
Among position means 57 §-859 .1028 

Among candidate scores 
grouped by positions 125 8.723 .06978 
Total 182 14.582 

.1028 F (05) =1.45 
F = 66978 = 1-473 or 1.5 F (or) =1.68 





This is as might be expected inasmuch 
as the rewards of the positions vary 
widely, with better candidates being 
nominated for the superior positions. 
The presence of significant positional 
differences has a crucial bearing on the 
evaluation of the variation between the 
scores of successful candidates and the 
scores of their unsuccessful competitors. 
The variation between the mean score 
for successful candidates and the mean 
score for unsuccessful candidates cannot 
be broken up into constituent variations, 
but the variation among candidate 


scores in the two groups (successful and 
unsuccessful candidates) can. It contains 
two elements, one, variation among can- 
didates for the same position, and, two, 
variation among position scores, and the 
latter variation is significant. Reference 
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has been made to the fact that success- 
ful candidates were to be compared, not 
with unsuccessful candidates in general, 
but with candidates who had competed 
unsuccessfully for the same positions. To 
reach this objective a method must be 
found for eliminating positional vari- 
ation from the base used for evaluating 
the differences between successful and 
unsuccessful candidates for the same po- 
sition. The differences among position 
scores are an extraneous element to com- 
parisons within positions. 

Table 2 shows what happens when 
the variation between successful candi- 
dates and unsuccessful candidates > is 
compared with the variation among 
candidate scores when the latter contains 
a variation among position scores as one 
of its constituent elements. ‘The denomi- 
nator of F, representing the variation 
among candidate scores, is spuriously in- 
flated and as a result the obtained F does 
not represent the comparisons desired. 
It is not surprising that the obtained F 
is not significant. 

Another objection to the method rep- 
resented by the result of Table 2 is that 
the variation between the two success- 
group means, represented in the numer- 
ator, is inaccurate because the candidate 
scores, of which one of the two means 
is made up, come disproportionally 
from the different positions. Since there 
is a significant variation among position 
means, the fact that the number of un- 
successful candidates varies from posi- 
tion to position is crucial. Thus the pro- 
portion of unsuccessful candidates from 
superior positions may have been greater 
than the proportion of superior positions 
in the total number of positions. If this 
had been true, it would have had the 
effect of reducing the differences between 
the successful and unsuccessful candi- 
dates, inasmuch as the number of suc- 


cessful candidates is uniform through- 
out the range of positions, being one to 
each, of course. 

What is required is a method for com- 
paring successful candidates with their 
unsuccessful competitors, position by 
position, thereby eliminating positional 
variation as well as the effect of unequal 
numbers of unsuccessful candidates 
among the several job-situations. This is 
provided for in a method suggested by 
Johnson and Neyman (11). For purposes 
of elementary exposition, the Johnson- 
Neyman procedure may be thought of as 
a method for effecting a pooling of com- 
parisons made within job-situations. All 
of the candidates for each position are 
classed in one or other of the eight sub- 
ject matter categories and therefore by 
eliminating positional variation, subject 
matter variation is also eliminated. The 
Johnson-Neyman method takes each in- 
dividual position in turn and compares 
the score of the successful candidate 
(which is also the mean) with the mean 
for his unsuccessful competitors, and 
evaluates the difference between these 
two means _in relation to the variation 


TABLE 4 
Relation of the variation between the score of the 
successful candidate and the mean of the scores of 
his unsuccessful competitors to an estimate of 
this variation based upon the variation among 
candidate scores unaffected by the variation 
among job-situation means 


For all 58 job-situations 





Sum of Mean 





Variation d.f. Squares Square 
Among job-situation 
means so. 5. BSS 
Between success-cate- 
gory scores I .281 .281 
Among candidate scores 


of the same job-situa- 
tion and the same 





success category 124 8.442 .06808 
Total 182 14.582 
281 F (05) =3.92 


F = 56808 =4-13 OF 4 F (or) =6.84 
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in candidate scores. This is done for each 
job-situation and the final F represents 
a pooling of the results. 

Table 4 indicates that for the entire 
group of 58 job-situations, the obtained 
F is significant at the 5 per cent level. 
The null hypothesis tested by this in- 
formation states that the two groups of 
scores, the candidate scores of successful 
candidates and the candidate scores of 
their unsuccessful competitors, are ran- 
dom samples from a homogeneous popu- 
lation. The obtained F is of a size that 
would occur about 5 times in 100 if the 
hypothesis were true. What conclusion 
regarding the population should be 
drawn from that known fact? Should the 
conclusion be that the event which did 
occur is unlikely under the null hypoth- 
esis and that therefore the hypothesis 
should be rejected and another hypoth- 
esis posited to explain the condition? Or 
should the conclusion be that while the 
event is unlikely, it is not so very un- 
likely, and the unusual did occur and 
therefore the hypothesis should be re- 
tained. The author leaves it to the reader 
to form his own judgment. Jf the null 
hypothesis is rejected the differences be- 
tween successful candidates and their 
unsuccessful competitors are to be con- 
sidered significant. If the null hypoth- 
esis is accepted, the differences are not to 
be considered significant. Most research 
workers would reject the null hypoth- 
esis. 

When the differences between the 
means, job-situation by job-situation, are 
considered, it is evident that in 34 of the 
job-situations the score of the successful 
candidates is lower (i.e., better) than the 
mean of the scores of the unsuccessful 
candidates, and in 24 job-situations 
higher. Inasmuch as sometimes the lower 
core favors the successful candidate and 


sometimes the mean for the unsuccessful 
candidates, an explanation is provided 
for the very small difference between the 
mean for all 58 of the successful candi- 
dates and the mean for all 125 of the un- 
successful candidates, a difference which 
is as small as .067. If the difference be- 
tween the score of the successful candi- 
date and the mean score for the unsuc- 
cessful candidates is computed for each 
job-situation individually, 58 differences 
are obtained. Disregarding signs, the 
mean of these 58 differences is .250, and 
the median is .217, both of which are 
much greater than .067. These are the 
differences which might presumably be 
evident to a reader of recommendations 
provided he had the set of recommenda- 
tions of the successful candidate and a 
composite set for the unsuccessful candi- 
dates and provided he did not know 
which was which. As can be readily seen 
the mean of the individual differences is 
small, being only one-quarter of an in- 
terval on the Quality Rating Scale, and 
does not warrant the assumption that in 
any individual case the successful candi- 
date can be distinguished from the un- 
successful candidates. The range of the 
differences is from .015 to .g99, and the 
standard deviation is .193. 

It is interesting to note that the me- 
dian for the 34 differences which favor 


the successful candidate is .24 whereas 


the median for the 24 differences which 
favor the unsuccessful candidates is only 
.16. The differences favoring the unsuc- 
cessful candidates are in no case greater 
than .529 whereas the differences favor- 
ing the successful candidate are as great 
as .g99. Of the 58 successful candidates, 
23 or 40% had the highest score of all of 
the candidates of their job-situations. 
Thirty candidates had scores that gave 
them a rank among the candidates of 
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their own job-situations above the me- 
dian position, and g had ranks at the 
median position." 

In Chapter 5 the effect of extra- 
recommendation factors was discussed 
and the nominees were classified as to 
whether their candidacy had been af- 
fected by such interfering factors. If 
either a candidate’s success or his failure 
could have been accounted for by such 
factors, he was considered to have been 
affected. In either case the candidate’s 
recommendation quality score does not 
represent his normal chances for place- 
ment success because the intrinsic rela- 
tion of the latter with recommendation 
quality has been interfered with by 
extra-recommendation factors. 

Among homogeneous job-situations 
unaffected successful candidates might be 
expected to have lower (i.e., better) 
recommendation quality scores than af- 
fected successful candidates inasmuch as 
unaffected successful candidates sup- 
posedly were successful because their 
recommendations were relatively good 
while affected successful candidates may 
have been successful because of the in- 
fluence of extra-recommendation factors. 
Unaffected unsuccessful candidates, on 
the other hand, might be expected to 
have higher (i.e., poorer) scores than af- 
fected unsuccessful candidates inasmuch 
as unaffected unsuccessful candidates 
supposedly were unsuccessful because 
their recommendations were relatively 
poor while the affected unsuccessful 
candidates may have been unsuccessful 
because of the adverse influence of extra- 


"Among five candidates arranged in the 
order of their scores with the best (lowest) scor- 
ing candidate first, the first and second candi- 
dates have ranks above the median position, 
the third candidate has a rank at the median 

sition, and the fourth and fifth candidates 

ave ranks below the median position. 


recommendation factors. And, in gen- 
eral, successful candidates might be ex- 
pected to have lower scores than un- 
successful candidates. This means that 
unaffected candidates might be expected 
to be found at the two extremes of the 
distribution of unadjusted means found 
in Table 5. The actual figures do not 


TABLE 5 


Unadjusted and adjusted means of candidate 
scores of groups classified in relation to the in- 
fluence r extra-recommendation factors ar- 
ranged in ascending order based upon the 
expected magnitude of the unadjusted mean, 
together with the data for the 
adjustment of the means 





Unad- Ad- 
Groups justed justed 
Means Means 





Unaffected successful 


candidates 2.391* (+.034) 2.425 
(N =42) 
Affected successful 
candidates 2.608 (—.125) 2.483 
= 16) 
Affected unsuccessful 
candidates 2.632 (—.125) 2.507 
(N = 23) 
Unaffected unsuccess- 
ful candidates 2.493 (+.034) 2.527 
(N = 102) 
General mean for all candidates, 
456.955 _ 
Mean for all unaffected candidates, 
ns 
Mean for all affected candidates, 
102.271 
—_——— = 2.622 
39 


General mean minus the mean for unaffected 
candidates, 2.497 —2.463 = .034 

Mean for affected candidates minus the general 
mean, 2.622—2.497=.125 





* Only one decimal place in the means is 
significant. 
seem to confirm the expectation. One 
analysis would conclude that the mean 
of the fourth group is inconsistent with 
the others. Another, and more fruitful 
supposition, however, is that the means 
of the second and third groups are in- 
consistent, Notice that both of these are 
affected groups. The original line of 
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reasoning posited homogeneous job- 
situations, but suppose there were re- 
liable differences between the job- 
situations in which extra-recommenda- 
tion factors operated and those in which 
they did not operate? As a matter of fact, 
such is the case, as will be shown by 
analysis of variance. But it is interesting 
to note that when the means are ad- 
justed for the difference between the 
mean for all unaffected candidates and 
the general mean, and for the differ- 
ence between the mean for all affected 
candidates and the general mean, the 
order of magnitude is precisely as might 
be expected. 

The study of the effect of extra- 
recommendation factors in Chapter 5 
left three classes of job-situations: 

1. Those in which either the successful 
candidate or all of the unsuccessful 
candidates, or both, were affected, leav- 
ing no pairs of competing candidates in 
which the relation was not subject to 
interference; 16 job-situations with 39 
candidates. 

Those in which none of the candidates 
were affected; 25 job-situations with 80 
candidates. 

3. Those in which one or more but not 
all of the unsuccessful candidates were 
affected and therefore had to be elimi- 
nated, but the job-situation was retained 
with whatever pairs of competing can- 
didates remained unaffected by extra- 
recommendation factors. By eliminat- 
ing 23 affected candidates in otherwise 
unaffected job-situations, 17 job-situa- 
tions were retained with 41 remaining 
candidates. 

Combining Classes 2 and 3 there are 

(2 job-situations with 121 candidates in 
which the relations of all of the pairs of 
competing candidates are free from the 
influence of extra-recommendation fac- 
tors. : 
It has already been stated that there 
ire reliable differences between job- 
situations in which extra-recommenda- 


ho 


tion factors may have operated and job- 
situations in which they were absent. 
This relation was analyzed with the 
Johnson-Neyman method for the 39 can- 
didates of Class 1 as one group and the 
121 candidates of Classes 2 and 3 com- 
bined as the other group. In order to 
eliminate the differences between suc- 
cessful and unsuccessful candidates from 
the comparisons between affected and 
unaffected candidates, the analysis was 
made by success groups. In other words, 
the scores of affected successful candi- 
dates were compared with those of un- 
affected successful candidates, and the 
scores of affected unsuccessful candidates 
were compared with those of unaffected 
unsuccessful candidates, and the results 
pooled. As is shown in Table 6 the dif- 


TABLE 6 


Relation of the variation between the mean score 
for candidates unaffected by extra-recommenda- 
tion factors and the mean score for candidates 
affected by extra-recommendation factors to an 
estimate of this variation based upon the varia- 
tion among candidate scores unaffected by the 
variation between success-category means 


For 58 job-situations which, with 23 of the 
original candidates eliminated, were internally 
homogeneous with respect to extra- 
recommendation factors 





Sum of Mean 


d.f. Squares Square 


Variation 





Between  success-cate- 





gory means I .077 
Between means for un- 
affected and affected 
candidates I 1.083 1.083 
Among candidate scores 
of the same category 
with respect to suc- 
cess and extra-recom- 
mendation factors 157 11.263 .071739 
Total 159 12.423 
1.083 F (05) =3.91 
= 7071739 15-10 OF 15 F (01) =6.81 





ferences between affected and unaffected 
candidates are highly reliable. The ob- 
tained F is significant at a point far 
beyond the 1 per cent level. 
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Because the differences between af- 
fected and unaffected candidates are so 
reliable, the differences between success- 
ful and unsuccessful candidates were 
analyzed for only the 42 job-situations 
with the 121 unaffected candidates. In 
this analysis the effect of extra-recom- 
mendation factors on the comparisons 
was removed, but at the expense of a 
very considerable reduction in the 
amount of data. Again the Johnson- 


TABLE 7 


Relation of the variation between the score of 
the successful candidate and the mean of the 
scores of his unsuccessful competititors to an 
estimate of this variation based upon the varia- 
tion among candidate scores unaffected by the 
variation among job-situation means 


For. 42 peers which, with 23 of the 
original candidates liminated, were unaffected 
by extra-recommendation factors 





Sum of Mean 





Variation d.f. Squares Square 
Among job-situation 
means 4I 3.310 
Between success-cate- 
gory scores I .2%4 .2%4 


— candidate scores 





of the same job-situa- 
tion and the same suc- 
cess category 78 4.646 .05956 
Total 120 8.170 
214 F (05) =3.096 
F =" 95956 73°59 or 4 F (01) = 6.06 





Neyman method was employed, the 
comparisons between successful and un- 
successful candidates being made posi- 
tion by position. The result is shown in 
Table 7. The obtained F is significant 
at close to the 5 per cent level. The dif- 
ference in significance between the F 
obtained with 58 job-situations (Table 
4) and this F is slight and does not re- 
quire any special explanation. However, 
it is interesting that the use of unaf- 
fected candidates exclusively did not pro- 
duce an F significant at a more extreme 
level of confidence. The reduction in the 
amount of data may explain this failure. 

This completes a description of the 
investigation into the relation between 
recommendation quality and placement 
success. Before a conclusion is reached 
it is well to remember that one can 
demand too extreme a level of signifi- 
cance and thereby overlook differences 
that are truly significant. The author 
believes that there is a reliable associa- 
tion of recommendation quality, as es- 
timated by the procedure used, with 
placement success, and that successful 
candidates have recommendations that 
are reliably better than those of their 
unsuccessful competitors. 
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CHAPTER 7 


CONCLUSIONS 


HIS INVESTIGATION may be described 
La a study concerned with the effec- 
tiveness of written recommendations in 
the placement process with the focus of 
attention upon the question of whether 
there are statistically significant differ- 
ences between the quality of the recom- 
mendations of succcessful candidates and 
the quality of the recommendations of the 
candidates who unsuccessfully competed 
for the same positions. The statistical 
analysis uncovered a reliable association 
between superior recommendation qual- 
ity, as defined in the investigation, and 
placement success (Tables 4, 7). But the 
fact of association does not establish a 
causal relation and the implications from 
the fact must rest upon a qualitative 
analysis of the relation studied. It is dif- 
ficult to conceive of superiority of recom- 
mendation quality as being other than a 
positive factor contributing to or helping 
to cause placement success. Recommenda- 
tions are written for the purpose of evalu- 
ating the individual as a candidate for a 
position and superior recommendations 
are intended to cause placement success. 
When, therefore, superior recommenda- 
tions are found to be associated with 
placement success, a natural conclusion 
is that they have helped to cause such 
success in the sense that candidates with 
superior recommendations are chosen 
partly because they have superior recom- 
mendations. Of course, it is recognized 
that superior recommendations have sig- 
nificance only because they are sympto- 
matic of superior qualifications possessed 
by the candidate. Thus, while within the 
framework of the placement process su- 
perior recommendations may be con- 
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sidered to cause placement success, in a 
more fundamental sense the cause lies 
deeper. Nevertheless, it is worthwhile to 
study the placement process itself and 
to attempt to trace causation within the 
process. Accordingly, and in that con- 
text, the author will state as his conclu- 
sion that there is association between su- 
perior recommendation quality and 
placement success, and that the investi- 
gation did not produce grounds for deny- 
ing the logical inference from a qualita- 
tive analysis of the relation, namely, that 
superior recommendation quality is a 
contributing cause of placement success. 
In other words, the investigation found 
that recommendations of superior qual- 
ity are effective in the process of secur- 
ing a teaching position. This refutes the 
charges of some members of the pro- 
fession that recommendations are, prac- 
tically speaking, of no consequence in 
the placement process. 

In addition, employers must be given 
more credit for discrimination in their 
selection of appointees than is some- 
times granted them. Undoubtedly pure 
luck is a large element of success in get- 
ting a position, but the fact that in the 
long run employers select the apparently 
more worthy candidates will come as a 
surprise to some people. 

The question of whether an estimate 
of recommendation quality is a valid 
measure of the candidate was not 
studied, but an incidental finding has 
a bearing on the problem. To be valid 
a system of estimates would have to dis- 
criminate reliably among the candidates, 
and correlate with a criterion. The re- 
sult shown in Table 1 indicates that the 
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estimate of recommendation quality dis- 
criminates reliably among the candidates 
and thereby satisfies the first requisite. 
Quite apart from the findings of this 
investigation, it is commonly believed 
that, in general, employers try to select 
the superior candidates for appointment. 
If that is true, and if the judgment of 
employers as evidenced in their appoint- 
ments is a satisfactory criterion, the sig- 
nificant association found between su- 
perior recommendation quality and 
placement success indicates a reliable 
degree of correlation with a criterion. 
This satisfies the second requisite. To 
that extent, then, the estimate of recom- 
mendation quality may be considered a 
valid measure of candidate quality. The 
reader is warned, however, that the cri- 
terion is far from adequate. 

Those that are skeptical of the validity 
and effectiveness of written, or descrip- 
tive, recommendations frequently offer 
as a substitute scaled ratings of the 
candidate on a list of traits. While in 
the controversy between the advocates 
of descriptive recommendations and 
scaled ratings the important considera- 
tion to bear in mind is that there is no 
practical or other reason why both can- 
not be used, there remains at least a 
theoretical interest in the question. The 
findings that recommendations reliably 
discriminate among candidates, that rec- 
ommendations seem to influence the 
employer, and that employers tend to 
choose candidates with recommendations 
of superior quality offer no support for 
the position of the proponents of scaled 
ratings. Obviously the findings of this 
study do not settle the question. 

Other findings incidental to the main 
investigation have important implica- 
tions. As is shown in Table 3 it was 
discovered that nominees for given posi- 
tions as groups differ reliably from one 


another. This indicates that the place- 
ment executive has done considerable 
selecting among candidates when nom- 
inations are made. Thus the recom- 
mendations have already served a useful 
function even before they are presented 
to the employer. This selection by nar- 
rowing the range of calibre of the candi- 
dates nominated reduces the likelihood 
of demonstrable differences between the 
recommendations of successful and un- 
successful candidates. That significant 
differences should have been found, 
nevertheless, is somewhat remarkable 
and a result of the precision of the sta- 
tistical method employed. 

The highly reliable differences found 
between candidates nominated in job- 
situations marked by extra-recommenda- 
tion factors and those not so marked as 
shown in Table 6 suggests questions 
which might well be the subject of 
further investigation. It would be in- 
teresting to know, for instance, if the 
positions of job-situations marked by 
extra-recommendation factors attract in- 
ferior candidates because they offer in- 
ferior rewards. The question would be 
especially interesting if it were investi- 
gated with respect to unreasonable re- 
quirements acting as interfering factors. 

The small mean difference between 
successful candidates and their unsuccess- 
ful competitors calculated for individual 
job-situations, one-quarter of an interval 
on the Quality Rating Scale, suggests 
that the procedure for analyzing recom- 
mendations and estimating their quality 
is not safe for the prediction of success 
or failure for an individual candidate 
in a given job-situation. 

One of the most important findings, 
shown in Table 1, is the demonstration 
that it is possible to construct a pro- 
cedure that will produce a score which 
reliably discriminates among candidates. 
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This means that a new field has been 
opened for research, which although 
relatively restricted, has important prob- 
lems remaining for attention. Other 
characteristics of | recommendations 
should be studied to discover if there 
is association between superiority with 
respect to the characteristic and place- 
ment success. Characteristics that come 
to mind are: 


1. Length of recommendation, or number 
of items commented upon not unfavor- 
ably. 

2. Proportion of comment upon impor- 
tant items as opposed to unimportant 
traits. 

3. Proportion of evaluation of the can- 
didate as opposed to information 
about the candidate. 

4. Quality of evaluation weighted for the 
importance of the items commented 
upon. 


o 


mented upon. 

Amount of advetse comment. 

Date of the recommendation. 

Reputation of the writer of the rec- 

ommendation. 

g. Degree and character of the acquaint- 
ance of the writer with the candidate. 

10. Degree and character of the acquaint- 
ance of the writer with the candidate’s 
teaching. 


DOI D 


[In addition, investigations might well 
be made of association between place- 
ment success and either comment upon 


Number of significant items not com- -. 
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individual .tems or quality of comment 
upon individual items. The findings of 
such a study would indicate the traits 
and qualifications that employers value 
on the basis of their actions. Hereto- 
fore there have been studies only of 
what employers say about traits and 
qualifications and their relation to teach- 
ing success. It may be that what employ- 
ers say on the subject is different from 
what they base their actions upon when 
actually selecting teachers. 

The principal lack in the study, as 
the author sees it, is that interviews 
were an uncontrolled factor. The prin- 
cipal merit lies in the fact that actual 
concrete job-situations were dealt with. 
Successful candidates were compared 
with the specific candidates who were re- 
jected by the same employer for the 
same position. The criterion of success 
is absolutely certain. The decisions of 
the employers studied were ‘not what 
they think they would have done in such 
and such circumstances, but what they 
actually did. Although the statistical 
procedure would have been greatly fa- 
cilitated if convenient numbers of candi- 
dates and job-situations could have been 
chosen, the authentic character of the 
data more than compensated for the 
loss. 
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APPENDIX 


OUTLINES OF JOB-SITUATION STUDIES 


Outlines of the individual studies of the 
job-situations referred to in Chapter 5 are 
presented in the following pages. 

1. In each study, the candidates, desig- 
nated by letters, are listed in alpha- 
betical order, the successful candidate 
invariably being listed first. Candidate 
A is always the successful candidate. 

2. Candidates eliminated from the job- 
situations are marked by an asterisk pre- 
ceding the letter designating the can- 


3. If the specification was stated in the 
inquiry as a preference rather than a 
requirement, the fact is indicated in 
the outline. 

4. When marital status, the number of 
children, and the degree held were con- 
sidered to be unimportant factors in a 
job-situation, information with regard 
to them is omitted from the tabular ar- 
rangement. 

5. The following special abbreviations are 


didate. used in the tabular arrangement: 
Abbreviation Meaning 

. That the candidate was eliminated from the job-situation. 

P Protestant. 

c Roman Catholic. 

o. k. Adequate subject matter preparation in the field in which the position is open. 

? Questionable subject matter preparation in the field in which the position is open. 


If the candidate’s field of primary specialization differs from the one in which 
the position is open, it is named. 


M That the candidate is married. 
The number of children the candidate has is given after the symbol for marital 
status. 
S That the candidate is single. 
No. 12 
Public school. 
English, grade 9. 
Sex requirement: Man. 
Specifications: Ability to assist in the coach- 
ing of football and basketball. Experience 
not required if the candidate has a strong 
scholastic record. 
Candidate Sex Age Religion Field Experience 
*A M 28 P 0. k. o 
* B M 24 P 0. k. 2 


Candidate A was without teaching experi- 
ence and had a definitely weak scholastic 
record judging from his recommendations, 
but he was able to coach football and basket- 
ball. Candidate B with 2 years of experience 
and a better scholastic record could coach 
only track and tennis. If the quality of the 
recommendations is not considered, presum- 
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ably ability to coach football and basketball 
outweighed both scholastic achievement and 
experience—despite the fact that the inquiry 
indicated that experience would be required 
unless the candidate had a strong scholastic 
record. 

The job-situation was eliminated, 
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No. 18 


Public school in a system widely known 
for its “progressive” outlook, organization, 


and program. 


Junior and senior high school speech, dra- 
matics, and public speaking to be integrated 
with the work of other courses. : 

Sex requirement: None. 


Specifications: None. 


Candidate Sex Age Religion Field Experience Marital Status Degree 
oA F 37 “Not a member English 19 S None 
of any” 
*B Seer ee o. k. 2 college M-—o M. A. 


Candidate A emphasized in the creden- 
tials her desire to teach in a “progressive 
school” and her sympathy with “progressive 
education.” 

Candidate A did not have a bachelor’s 
degree at the time she was employed, but the 
credentials indicated that she would receive 
the degree in the February following the 
September in which the work of the position 
was to begin. (She actually received the 
degree a year after that time.) This candi- 
date’s training in the field of speech was 
weak, noticeably so in contrast to that of 
Candidate B, but her experience had been 
in the very area desired, the integration of 
speech with other activities and subject mat- 
ter materials. 


Candidate B had had much more formal 
training in the field of speech and, in addi- 
tion, as was emphasized in her credentials, 
valuable experience in the professional thea- 
tre. She had neither the quantity nor quality 
of teaching experience of Candidate A, 
however. 

Candidate B was married, but it is un- 
likely that in this system that fact would 
have been to her disadvantage. 

Because of the difference in quantity and 
quality of experience of the two candidates 
and the difference in attitude toward “pro- 
gressive education,” it was decided that 
there might have been interference. 

The job-situation was eliminated. 


No. 22 


Public high school. 

Spanish for advanced students. 
Sex requirement: None. 
Specifications: None. 


Candidate Sex Age Religion Field 

A F 39 “None” o. k. 
whe. 7 ? 

Cc F 43 “Non-member” o. k. 


Experience Marital Status 


134 M—o 
o *¢ S 
16 S 


** Candidate B had had 5 weeks of substitute teaching experience. 


Candidate B was eliminated because of her 
weak preparation and lack of teaching ex- 
perience. 

Candidate A had had more teaching ex- 
perience in Spanish than Candidate C, but 
the latter was qualified to teach advanced 
high school students in Spanish even though 
her major was French. Her experience in- 
cluded 4 years of teaching freshman and 
sophomore Spanish in a teachers college ‘as 
well as critic teaching in Spanish in the 


training school. Her preparation in Spanish 
in terms of graduate. courses was almost as 
thorough as that of Candidate A. 

It seems unlikely that the difference of 
4 years in the ages of the two candidates was 
a factor. 

Note how each of these two candidates re- 
plied to the question regarding church affili- 
ation. 

The job-situation was retained with Can- 
didates A and C. 


Spied. kien 


. Ra oS 
= éSeey- Foe 








ae 
ee, 


net aS Te 


’ 
RY st mee pe 
2 AO. mt 








Tae 


~ 
_ 
we 
rN er — me 
7“ 











62 WELLS HARRINGTON 


No. 24 


Public school. 


Latin, 4 classes, and Italian, 1 class; high 


school level. 


Sex requirement: None. 


Specifications: None. 


Candidate Sex Age Religion Field Experience Marital Status 
* A M 26 C History oO S 
* B 7: SS o. k. oO S 
Cc F 36 C o. k. o S 


The credentials of Candidate A, a “local 
candidate,” were requested by the employer. 
If the list of courses, undergraduate and 
graduate, presented in the candidate's jcre- 
dentials is accurate, he could not properly 
have obtained a certificate to teach Latin in 
the state in which the position was located, 
although he had had 16 semester hours of 
undergraduate work in Latin. He had had no 
training in Italian, and there is no evidence 
that he was of Italian extraction or that he 
was brought up in a home where Italian 
was spoken. The transcript of his under- 
graduate work on file in the Registrar’s Of- 
fice had been sent to the state department 
of education involved so the accuracy of the 
lists of courses presented in the credentials 
could not be checked, but it seems reason- 


able to assume that the candidate was not as 
well prepared to teach Latin as his com- 
petitors. Candidate A stated that he was pre- 
pared to accept positions in “history, English, 
social science, and also Latin, elementary 
mathematics, and grammar grade work.” 
The report on the position returned by the 
candidate indicates that he actually taught 
Latin and English while in the position. 
Presumably the extra subject to be taught 
was changed to suit the candidate’s prepara- 
tion. Without the quality of the recom- 
mendations being taken into account, this 
seems to be a case in which a “local candi- 
date” was given the position in spite of the 
other considerations. 
The job-situation was eliminated. 


No. 37 


Public high school. 


Social science. 


Sex requirement: Man. 

Specifications: Experience preferred. Abil- 
ity to assist with the coaching of football, 
but not necessary if of good calibre other- 


wise. 
Candidate Sex Age Religion Field Experience 
A M 37 P o. k. Oo 
°* 8B M 24 Cc o. k. o 
Cc M 28 P o. k. 1 college teaching of German 
5 supervision of recreation 
A Had had no experience and apparently no ability in athletics. 
B Could coach athletics, . 
C Had had extensive experience in the direction of athletics and recreation. 


Candidate B, who stated an interest in the 
high school age group, was eliminated be- 
cause of the possibility that there might be 
prejudice against his religious affiliation. The 
Office of Placement Service executives did 
not know the community or the employer 
well enough to be confident that religion 
would not operate as a factor in this job- 


f 
J 
} 


situation. 

Apparently the preferences listed in the 
inquiry really operated as preferences rather 
than as requirements. The successful candi- 
date was not experienced either in teaching 
or in the direction of athletics. 

The job-situation was retained with Can- 
didates A and C. 
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